Category: Python
-
Python generator frames: Inspecting locals with gi_frame
When a Python generator yields, its stack frame remains — inspect it for the current line and local variables:
-
MIT just called for an educational revolution
As someone who teaches Python programming for a living, I’ve spent the last few years wrestling with the educational implications of AI. I’m changing everything I do to adjust to our new AI reality, experimenting with new ideas, including my AI-based Socratic tutor (https://practice.lernerpython.com/). I keep what works, throw away what doesn’t, and then try…
-
AI seminars — just one of the big changes at LernerPython
July and August are often when people take a break or go on vacation. But for me, this summer has been super busy, full of writing and improving LernerPython based on feedback I’ve gotten from people around the world. And so, I’m here with some big changes I’m making at LernerPython World Headquarters: 1. AI…
-
PyArrow dtypes in Pandas aren’t always faster than NumPy
Using PyArrow dtypes in Python Pandas isn’t always faster: %timeit s_pyarr.mean() # 21.5ms%timeit s_np.mean() # 47.3ms %timeit s_pyarr.nlargest(10) # 667ms%timeit s_np.nlargest(10) # 663ms
-
PyArrow strings in pandas: Faster and smaller than object dtype
PyArrow strings in Python Pandas are smaller than Python strings. But they’re also far faster: %timeit s_pyarr.str.len() # 106 µs%timeit s_py.str.len() # 1.6 ms In many examples, PyArrow was far faster.
-
PyArrow dtypes in pandas: Reading CSVs with dtype_backend
Want PyArrow dtypes in your Python Pandas data frame? The dtypes are double[pyarrow], int64[pyarrow], and string[pyarrow], not the normal NumPy ones. Note: This is still experimental… but it’s also the future.
-
Pandas 3 string dtype: PyArrow strings cut memory use
Using Python Pandas 3? Strings use PyArrow, not Pandas 2’s Python strings (dtype “object”):
-
PyArrow dtypes in Pandas 3: Missing values without float conversion
In Python Pandas 3, you can use PyArrow dtypes — which are nullable: What is s? 0 101 202 <NA> # pd.NA, not np.nan3 40dtype: int64[pyarrow] # see? Not float!
-
Pandas missing values: Why assigning pd.NA gives you float64
Assign either np.nan or pd.NA to a Python Pandas series with a NumPy dtype, and it’ll be np.nan, a float. Which means the entire series has a dtype of float: Result: 0 10.01 20.02 NaN3 40.0dtype: float64
-
Pandas int8 overflow: Why small dtypes wrap around silently
If your dtype is too small, operations on your Python Pandas series will fail: Returns: 0 1101 1202 -126 # 🤯dtype: int8 This does give an error: s + 500OverflowError: Python integer 500 out of bounds for int8