Question 10 of 20 Fundamentals Interpretation
Why does a pandas integer column turn into floats when a value is missing?
Short answer
The default NumPy int64 dtype has no missing-value marker, so pandas uses NaN, which is a float, and the whole column becomes float64. The nullable Int64 dtype keeps integers and marks gaps with pd.NA.
Why
When a column of plate numbers or read counts gains one missing value, for example after a reindex or a left join, pandas has to store something in the gap. NumPy integers cannot hold NaN, so the column is cast to float64 and 12 now prints as 12.0. Usually that is cosmetic, but it can do real harm.
An identifier stored as float can lose digits, because float64 represents integers exactly only up to 2 to the power 53, and the pandas guide warns that some integers cannot be represented as floats at all. A merge key that changed from int to float may also stop matching a table that kept integers. A boolean column with a gap becomes object, which breaks filters.
The remedy is the nullable dtypes. Ask for dtype="Int64" with a capital I, or call convert_dtypes(), and the gap becomes pd.NA while the values stay integers. pd.NA follows three-valued logic, so pd.NA == 1 returns NA rather than False, and you test for it with isna(). Nullable dtypes do not fix the upstream question of why the value is missing, and some older libraries still expect plain NumPy arrays and convert NA back to NaN or object.
A strong answer covers
- NumPy integers cannot store NaN
- NaN is a float, so the column is cast to float64
- Large identifiers can lose precision as floats
- Use the nullable Int64 dtype or convert_dtypes
- Test missingness with isna, not equality
Common mistakes
- Treating 12.0 in a count column as a parsing bug instead of a sign that a missing value entered the column
- Testing for missing values with == np.nan, which is always False
The interviewer might ask next
How would a float-typed sample identifier break a later merge with a LIMS export?