Category: Python
-
Pandas groupby with sort=False: Keep groups in order of appearance
A “groupby” call in Python Pandas is normally sorted by index. But if you’re grouping by month name, April will come before January. Pass “sort=False”, and the index will reflect the order of appearance:
-
Pandas groupby basics: Aggregating a numeric column by category
The simplest grouping in Python Pandas is groupby: For example: Returns a series whose index is the unique values from passenger_count.
-
Pandas dropna with thresh: Drop only rows with too many NaN values
You can remove NaN from a Python Pandas data frame with dropna, but be careful: It removes rows with even one NaN, which can be overkill. Pass “thresh” to allow some rows to remain, even if they contain NaN:
-
Pandas dropna: The best way to remove NaN values from a series
Coming to Python Pandas from NumPy? You’ll reach for np.isnan: Unfortunately, this works. Better, use s.isna (or s.isnull). But the best way to drop NaN? Use dropna:
-
New on the LernerPython practice system: A visual debugger
I’m a big fan of exercises, which is why the LernerPython platform includes hundreds of them — all using my in-browser practice system, which handles Python, Pandas, and Git, along with an AI-based Socratic tutor based on my writing. As of today, the practice system is even better: It includes a visual debugger, allowing you…
-
PyArrow dtypes in pandas: Nullable integers with pd.NA
PyArrow dtypes in Python Pandas are nullable (with pd.NA): s is: 0 101 <NA>2 30dtype: int64[pyarrow] The dtype is int64, but allows nulls. (Use np.nan? It’s turned into pd.NA.)
-
Pandas nullable integers: Keeping ints with Int64 and pd.NA
You have a Python Pandas series with ints + NaN. You don’t want float forced on you. Solution: Use the “extension” type Int64 (note Initial Caps) and pd.NA: s is: 0 101 <NA>2 30dtype: Int64
-
Get Python help, one on one: my coaching sessions
Some of the most satisfying work I do happens one on one. You arrive with a real problem from your real job — code that will not behave, an architecture you are unsure about, a Git situation that has you stuck — and an hour later it is smaller, or gone. I have offered these…
-
NaN in Pandas: Why you need np.nan, not pd.NaN
Missing data? NumPy calls it nan. Python Pandas displays it as NaN. But: Pandas doesn’t define pd.nan or pd.NaN. NumPy removed np.NaN in version 2.0. So you have to refer to np.nan from within Pandas, and it’ll be displayed as NaN.
-
np.nan in Pandas: Why missing values break comparisons
Missing data in Python Pandas? We use nan (“not a number”), which comes from NumPy. np.nan is a float, but not a normal one: