Why might a pandas groupby summary silently leave out rows?

Why

Suppose a table of 1,000 assay results is grouped by batch and 40 rows have no batch recorded. df.groupby("batch").size() returns counts that sum to 960. The missing 40 rows are often the most interesting ones, such as samples whose metadata failed to load. The pandas reference states that with dropna=True, rows whose key is NA are dropped, and with dropna=False they form their own NaN group.

Categorical keys add a second effect. The observed parameter decides whether categories with no rows appear in the result, and pandas 3.0 changed its default to True, so empty categories such as a planned dose level with no samples no longer appear as zero rows. Code written for pandas 2 that relied on seeing those zeros will now get a shorter table.

Aggregations add a third layer, because mean and sum skip NaN values within a group by default, so a group mean can rest on fewer values than its size suggests. The fix is to check that the group counts sum to the input row count, pass dropna=False while exploring, and report count alongside mean.