Introduction to the Python Developer Tooling Handbook
👋 Hey, I’m Tim Hopper! I’ve been a professional Python developer for over a decade. Like many Python developers, I’ve faced the many challenges of Python packaging and dependency management. I’ve also come to love helping developers find the right tools to make their work easier and more productive.
I am currently writing a free ebook on Python developer tooling. This isn’t a book about Python programming. Instead, the goal of this book is to help you understand the ecosystem of tools used to make Python development easier and more productive.
For example, this book will help you make sense of the complex world of building Python packages: what exactly are Poetry, Flit, Setuptools, and Hatch? What are the pros and cons of each? How do they compare to each other? It also covers tools for linting, formatting, and managing dependencies.
This handbook covers a wide range of other topics:
- Linting: What’s the difference between flake8, pylint, and ruff? What do you recommend?
- Type Checking: What’s the difference between mypy, pyright, and pyre? Which should you choose?
- Formatting: Why is everyone talking about ruff-format? How do I get started with it?
- Dependency Management: Overwhelmed by pip, poetry, and conda? I’ll help you make sense of it all.
- Virtual Environments: What are they and why should you use them?
If there are any topics you’d like to see covered in the handbook, please use the feedback form.
Blog Posts
uv 0.3: One command line to rule them all (almost)
Yesterday, Charlie Marsh and his team at Astral released uv 0.3.0. For those who may have missed it, uv was released earlier this year primarily as a blazing-fast replacement for pip for installing Python packages. This new release provides new documentation and a host of new functionality. Charlie Marsh and Simon Willison have written helpful posts about the new functionality. I thought it’d be helpful to illustrate how the current uv functionality compares to existing tools:
August 21, 2024
Require pip to install packages in virtual environment
A frequently heard piece of advice in improving Python development practice is using virtual environments to install dependencies. Virtual environments isolate your dependencies per project to protect you from annoying version conflicts. You can restrict pip to only install in virtual environments by creating a pip config file and setting: [global] require-virtualenv = True Mac and Linux users can put this config in $HOME/.config/pip/pip.conf and Windows users in %APPDATA%\pip\pip.ini. If you need to install a package globally for a command line tool (like csvkit), I recommend using pipx.
March 4, 2024
uv got to keep up: a new installer announced!
Astral, Charlie Marsh's company best known for Ruff, announced uv today, 'an extremely fast Python package installer and resolver, written in Rust.'
February 16, 2024
Pip and Poetry and Hatch, Oh My!
Modern Python developers have a plethora of tooling available to help the development process. When I started with Python in 2011, packaging was done with setuptools (or its predecessor distutils), and virtual environments were created with the virtualenv library. Now developers can (have to?) choose between a breadth of packaging tools (e.g., setuptools, Flit, Poetry, PDM, Hatch). Some of these tools manage virtual environments while others require the developer to do this themself.
February 14, 2024
Sponsor the Python Developer Tooling Handbook!
I’d like to release the first edition of the Python Developer Tooling Handbook in 2024. This project is a labor of love from me to the Python community. If you’ve ever struggled with Python packaging or dependency management, please consider sponsoring the project on Github.
February 14, 2024
Scientific Python Library Development Guide
Along with the Python Packaging User Guide, another excellent resource for learning about Python development practice is the Scientific Python Library Development Guide, a “guide is maintained by the scientific Python community for the benefit of fellow scientists and research software engineers”. I especially like their topical guides on topics like packaging and getting started with Github Actions.
February 12, 2024
Python Packaging Tool Examples
In the last few years, Python packaging has seen a lot of change. Instead of every project using setuptools/setup.py, there are now many tools to choose from when building and distributing Python packages. This can be overwhelming for new and experienced developers alike. My goal with the Python Developer Tooling Handbook is to help you make informed choices to make your Python development easier and more productive. As I’ve tried to learn the ins-and-outs of these tools, I created a series of example repositories showing how to use each of the major packaging tools.
February 6, 2024
Quick start guide for Python development on a Mac
Python developers (and hobbyists) often hear the advice not to use the preinstalled “system” Python on a Mac. When looking into alternatives, it’s easy to get overwhelmed by all the options: python.org installers, Homebrew, pyenv, conda, pyflow, rye, and more. My forthcoming ebook is designed to help you make informed choices about Python development and let you focus on writing code instead of managing your development environment. In the meantime, I’ll point you to Dylan Stein’s excellent Quick start guide for Python development on a Mac.
February 5, 2024
Python Packaging User Guide
The Python Packaging Authority produces an excellent resource called the Python Packaging User Guide, “a collection of tutorials and references to help you distribute and install Python packages with modern tools”. If you’re feeling frustrated by Python packaging, I highly recommend reading through this guide and considering it’s suggestions. My forthcoming Python Developer Tooling Handbook will cover much of the same material, but it intended as a complement to the Python Packaging User Guide, not a replacement.
February 3, 2024
Introduction to Rye
Armin Ronacher’s Rye is an exciting new tool in the Python packaging and dependency management ecosystem. Rye attempts to be a “one-stop-shop” to “to install and manages Python installations, pyproject.toml based projects, dependencies and virtualenvs seamlessly”. You can read more about the Rye philosophy here. Armin recently recorded an introduction to the tool with the release of version 0.21:
February 3, 2024