Why isn't Python packaging part of core development?
bitecode.dev just posted a massive video interview with Python core dev Brett Cannon. Among many other things, Brett explains why Python packaging hasn’t been part of Python core development:
Basically, they’re separate because Guido doesn’t care about packaging. That’s really what it comes down to. When packaging started to become a thing, we just didn’t have any interest in it, and so it never became a core dev concern. It just never came up—it was just not our thing. So the community in various ways stepped up to fill that void.
It became an area of expertise that the core devs just didn’t have. People didn’t become core devs because of their packaging knowledge. People became core devs because of them working on stuff in the standard library or the interpreter or whatever. So having packaging knowledge just doesn’t win you points for being a core dev, really, because it just doesn’t play into it because of the separation. And once again, because it just wasn’t interesting.
The community came up with their own solutions, whether that was easy_install and setuptools and whatever. We tried to be honest with distutils and it was okay, but it wasn’t ever great and it was hard. Once again, because the core devs didn’t really make that a priority of something that they felt they should own, it never really grew. Plus, changing anything in the standard library is hard because everyone automatically gets mad because it gets so widely used—hence why distutils got removed.
It was just one of these things. It just never became a thing, and so separate groups ended up taking ownership of it. Ian Bicking came up with pip and all that, and it just got widely used enough that it was the thing everyone used. People were using distutils, but setuptools got pretty popular and became kind of the de facto thing that people use. But that also got so big that people were too afraid of changing it because it would break so much stuff. And so it was just a whole thing.
It just developed in its own little group. And then there’s Conda over doing its thing as well in the scientific community. So it just never became a core thing. It’s always been run as kind of an independent group of people who have some overlap. Don’t get me wrong—there’s a bunch of core devs who are in both camps to some degree, to varying degrees, so it’s not like there’s no communication. It’s just packaging is not run by the core development team. It’s run by a separate group of people, whether you’re talking about the PyPI side or Conda doing its own thing and all that. It’s just always been separate groups. Just organically ended up that way.
Be sure to listen to the whole podcast to learn:
- Brett’s Python history
- How consensus-building works among core developers, handling syntax debates like assignment expressions
- Challenges of managing expanded community feedback, criticism, and the need for codes of conduct
- Brett’s goal to reimplement CPython in Rust