Data Analyst Interview
Menu
Browse in your language. All mock interviews, preparation sessions and feedback are in English only.

Why pandas Chained Assignment Leaves Your Data Unchanged

With pandas Copy-on-Write enabled, chained assignment such as df[‘amount’][mask] = 0 does not update the original DataFrame. Assign through the target DataFrame in one step instead: df.loc[mask, ‘amount’] = 0. Then verify the selected values and the rows that should remain unchanged.

A correction that leaves the report unchanged

A fictional analyst is testing a correction workflow for a US, UK, and German order report. Four orders contain USD-normalized values of $120, $80, $100, and $60. The exercise instructs the analyst to replace the two US values with zero. This is a deliberately simple test rule, not a recommendation to zero revenue by geography or an accounting policy.

The original total is $360. The selected US rows contribute $180, so the corrected test total should be $180. If the report still shows $360, the analyst needs to investigate whether the assignment reached the intended DataFrame.

Reproduce the failure and verify the fix

This script was tested with pandas 2.3.3 and Copy-on-Write explicitly enabled. The official Copy-on-Write guide documents the default behavior introduced in pandas 3.0. Older versions and modes can behave differently, so record your installed version rather than relying on an old notebook’s warning behavior.

import warnings
import pandas as pd

# Required for this pandas 2.3.3 reproduction.
# Copy-on-Write is the default in pandas 3.0.
pd.options.mode.copy_on_write = True
orders = pd.DataFrame({
    'order_id': ['US-1', 'UK-1', 'DE-1', 'US-2'],
    'market': ['US', 'UK', 'DE', 'US'],
    'net_usd': [120, 80, 100, 60],
})
original = orders.copy()
mask = orders['market'].eq('US')

bad = orders.copy()
with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter('always')
    bad['net_usd'][mask] = 0
assert bad.equals(original)
print('unchanged after chained assignment:', bad['net_usd'].sum())

fixed = orders.copy()
fixed.loc[mask, 'net_usd'] = 0
assert fixed['net_usd'].tolist() == [0, 80, 100, 0]
assert fixed.loc[~mask].equals(original.loc[~mask])
assert orders.equals(original)
assert fixed['net_usd'].sum() == 180
print('corrected total USD:', fixed['net_usd'].sum())
print('matched rows:', int(mask.sum()))

What does the output prove?

The tested output is unchanged after chained assignment: 360; corrected total USD: 180; matched rows: 2. The first assertion confirms that the attempted chained update left the entire bad DataFrame equal to the original. The corrected values are [0, 80, 100, 0].

The remaining assertions check that UK and German rows retain their original values and that the source orders DataFrame stays unchanged. A total alone is insufficient: two wrong row updates can offset one another and still produce a plausible sum. Here the expected row values and selection count make the intended result explicit.

Why does one .loc assignment help?

The chained expression first selects a column and then assigns through that intermediate object. Under Copy-on-Write, modifying that selection does not propagate back to the original DataFrame. The corrected expression names fixed as the assignment target and supplies its row mask and column together.

The DataFrame.loc reference documents boolean row selection and setting values. In this fixture, mask was created from orders and fixed preserves exactly the same index. In a pipeline that filters, reorders, or replaces the index, rebuild or validate the mask against the actual target before assignment.

Should you suppress the warning?

The demonstration captures warnings only around the deliberately incorrect line so both paths can run in one reproducible example. It does not use the absence of a warning as evidence of success. Production code should remove the chained assignment and test the intended data change instead of hiding warnings globally.

The copies separate the unchanged source, failed attempt, and corrected result for comparison. If the intended output is a separate report, create it explicitly and continue using that report. If the intended output is an update to orders itself, the assignment target should be orders.loc[mask, ‘net_usd’].

What should you inspect in a real correction workflow?

Verify the business key, rule, and expected selection size before changing values. This exercise assumes unique order IDs, known market labels, valid integer dollar values, and no missing amounts. Real data needs explicit handling for missing market codes and currency conversion. Keep original values and the correction reason in an audit trail when the report requires traceability.

Distinguish a failed assignment from an alignment problem. If a Series on the right side introduces unexpected missing values, use the separate Series assignment and index alignment example. If selecting a mask and its inverse loses records, inspect nullable boolean masks. Those require different checks from the Copy-on-Write issue shown here.

A concise interview explanation is: “I choose the DataFrame I intend to change, assign rows and columns in one .loc operation, and verify the selected rows, unselected rows, and total. I record the pandas version and keep an unchanged comparison dataset so a plausible aggregate cannot hide a failed correction.”

Leave a Comment

✉ WhatsApp