Why Does Assigning a pandas Series Create NaN Values?
Check index labels and order identity before treating a missing result as a missing source value.
By Prakhar Shrivastava · October 15, 2026
Assigning a Series to a DataFrame column matches index labels, not row positions. If a destination label has no matching source label, the assigned result is missing. Equal row counts do not establish that the labels match or that the rows describe the same orders.
Reproduce missing values with two orders
Consider a fictional US and UK reporting extract with one row per order. The amounts are already in USD under the reporting policy. Filtering an earlier table has left row labels 10 and 20. A separately created Series has default labels 0 and 1.
import pandas as pd
orders = pd.DataFrame(
{'market': ['US', 'UK'], 'order_id': [101, 202]},
index=[10, 20]
)
amounts = pd.Series([75.0, 50.0])
orders['unmatched_usd'] = amounts
print(orders['unmatched_usd'].isna().tolist())
# [True, True]The source amounts exist, but neither source label matches a destination label. This is an alignment failure, not evidence that both orders have unknown revenue. Replacing the resulting NaN values with zero would hide the error and turn a $125 source total into a zero-value report.
Prefer an explicit business key when sources are separate
If a source lookup is keyed by order ID, use that identity directly. In this fixture the lookup deliberately lists order 202 before order 101.
by_id = pd.Series([50.0, 75.0], index=[202, 101])
assert by_id.index.is_unique
orders['revenue_usd'] = orders['order_id'].map(by_id)
print(orders['revenue_usd'].tolist())
# [75.0, 50.0]Order 101 receives $75 and order 202 receives $50 regardless of lookup order. This assumes order IDs are unique across this lookup. If identifiers are only unique within a market or store, use the full business key in a validated merge. Audit missing keys and unmatched records separately; a missing mapping is not automatically a zero amount.
Index alignment can also be exactly what you want
When both objects retain the same meaningful row labels, their display order need not match. The reversed Series below still assigns the correct amounts:
by_row = pd.Series([50.0, 75.0], index=[20, 10])
orders['aligned_usd'] = by_row
print(orders['aligned_usd'].tolist())
# [75.0, 50.0]Label 10 receives the value stored under label 10, even though it appears second in the Series. A row index is useful for alignment only when its meaning has been preserved. Two unrelated exports can both have indexes 0 and 1 while listing different orders.
Why to_numpy is not an automatic repair
orders['positional_usd'] = by_row.to_numpy()
print(orders['positional_usd'].tolist())
# [50.0, 75.0]Removing labels makes assignment positional. It also swaps the intended amounts in this fixture: the US order now receives $50 and the UK order $75. The total remains $125, so a sum-only check misses the error. Equal lengths and matching grand totals are insufficient to establish correct attribution.
Use positional assignment only when the source contract guarantees the same ordered records and that guarantee has been checked. Resetting indexes on both objects similarly replaces identity with position; it can remove the visible symptom without proving the rows correspond. Reindexing a Series to the destination labels retains label-based matching and cannot invent missing identities.
What should a reporting check verify?
Inspect both indexes, key uniqueness, unmatched keys, and a few known order-to-amount pairs. Preserve the original order ID and market through transformations. If the input is line-level, establish order totals before assigning them to an order report; see the average-order-value example.
Five fixture checks passed in pandas 2.2.3: all-missing assignment for disjoint labels, the keyed lookup result, reversed-label alignment, the incorrect positional sequence, and the identical $125 totals despite swapped attribution. Run equivalent checks in your deployed environment and include filtered or reordered inputs.
For group statistics that must retain input rows, the transform versus agg guide covers a related but different operation. For joins, check null-key attribution. Primary references are the pandas indexing guide and DataFrame reference. The practical goal is a correct record-to-value mapping, not merely a column without NaN values.