To run Python code you need to ... 1. Choose the appropriate/best language version 2. Find/install an interpreter to meet that language version need 3. Create a virtual environment that uses the selected interpreter 4. Install the code's dependencies 5. Run the code # Goals - [ ] Easiest way to [[#`py launch`|launch an interpreter]] - [x] Select the newest version - [x] Specify the version - [ ] Support the major OSs - [x] Linux - [x] macOS - [ ] [[#Windows support]] - [ ] Registry - [ ] Microsoft Store - [x] From a virtual environment - [ ] Install an interpreter (requires [[Relocatable builds]]) - [ ] [[Virtual environment location]] - [ ] [Inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/) for `requires-python` only? - [ ] Easiest way to [[#`py list`|list interpreters]] - [ ] Fancy/human-readable - [ ] JSONL - [ ] Easiest way to [[#`py run`|run code]] 🏃 - [ ] [Inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/) - [ ] [[requires-python spec]] - [ ] `.py` files - [ ] Path - [ ] URL - [ ] Directories via `__main__.py`? - [ ] Zip files via `__main__.py`? - [ ] `pylock.toml` (takes care of interpreter selection and virtual environment creation, even if you do specify config details) - [ ] Scripts next to `pylock.toml`? - [ ] Directories w/ `__main__.py`? - [ ] Modules (`-m`) based on cwd? - [ ] `bin/` based on cwd? - [ ] Zip file? - [ ] `pyproject.toml` (takes care of interpreter selection and virtual environment creation, even if you do specify config details) - [ ] Write out lock file? - [ ] Scripts? - [ ] Directories w/ `__main__.py`? - [ ] Modules (`-m`) based on cwd? - [ ] Zip file? - [ ] `bin/` based on cwd? - [ ] Install Python on demand (requires [[Relocatable builds]]) - [ ] `py x` shorthand (like from `chmod`, plus nod to pipx)? - [ ] `venv` + `sync` for more manual control while keeping installation of packages simple (requires way to launch interpreter only) - [ ] Easiest way to [[#`py tool`|run tools]] 🧰 - [ ] Install - [ ] Run (`py x`?) # Ideas ## Automatic virtual environment management The goal is to make virtual environments an implementation detail to help prevent users from messing up their install and to simplify their workflows. There are two situations to consider: _ephemeral_ environments (like for inline script metadata), and _long-lasting_ environments (e.g. `.venv/` in a project). Probably need to make sure that if they manage to install something into an ephemeral virtual environment it's considered different than when it was first created. Simplest thing would be to create a single-use lock file for the environment and verify that everything matches. The next possibility that's even more robust is to keep projects unpacked somewhere, have them be read-only and recreate the virtual environments from scratch every time via filesystem links. Determining which ephemeral virtual environment to use could be done by hashing the inputs used to do the install. Ephemeral virtual environments should be occasionally cleaned up. Maybe every 30 days or past the same day of the month the following month? It could be done in a background thread for all ephemeral virtual environments. There should also be a way to forcibly recreate the virtual environment to make sure to use the newest project versions. ## Subcommands [Proposal for user-contributed subcommands](https://github.com/brettcannon/python-launcher/discussions/222) - Make each command a feature of the crate to control what commands are available? - Complex commands should decompose to other commands ### `py launch` (or `py exec`) - [`pymanager exec`](https://peps.python.org/pep-0773/#exec-subcommand) Same as bare `py` in terms of semantics/purpose (or skip and push people towards [[#`py run`]]?). Workflow tools do not have an equivalent command. Everyone else has their `run` command execute the command as found in the virtual environment if a path is not provided, i.e. `run python`. - `--version`/`--python`: specify the Python version, much like e.g. `-3.12` - `uv` uses `--python` - `--ignore-venv`/`--exclude-venv`/`--skip-venv`: exclude virtual environments from the list of potential interpreters ([poll on flag name](https://fosstodon.org/@brettcannon/114394209663606574)) - `--`: All flags following are passed on to the interpreter - Used by [`uv run`](https://docs.astral.sh/uv/reference/cli/#uv-run) - `-m`: Equivalent to `-m` to the interpreter - Pragmatic shortcut to minimize need to specify `--` for the `-m` common case - Argument parsing stops once a path, `-m`, or `--` is found ### `py run` - [`pdm run`](https://pdm-project.org/en/latest/reference/cli/#run) - [`poetry run`](https://python-poetry.org/docs/cli/#run) - [`hatch run`](https://hatch.pypa.io/latest/cli/reference/#hatch-run) - [`uv run`](https://docs.astral.sh/uv/reference/cli/#uv-run) - [`pipx run`](https://pipx.pypa.io/latest/docs/#pipx-run) For simplest way to execute code in an implicit virtual environment (i.e. the Python interpreter is an implementation detail). Need to consider [[Caching app files while being ignored by backup apps]] for the implicit virtual environments. - `--group`/`--with`: Specifies the dependency group to use (only applicable when a directory path is specified, maybe zip file if it ships with a `pyproject.toml`) - Care about extras, e.g. `--extra` or implicitly specified via `--group`? - `uv` uses `--extra` - If a file path is specified: - If the file is a zip file, do anything special like look for inline script metadata in `__main__.py`? - If [inline script metadata](https://packaging.python.org/en/latest/specifications/inline-script-metadata/) is specified, use that - Otherwise assume metadata of no dependencies and any Python version - If a directory path is specified: - For convenience when a [directory contains a `__main__.py` file](https://docs.python.org/3/using/cmdline.html) - Use inline script metadata or `pyproject.toml`? - Provide a way to specify whether `launch` or `run` are the default when no subcommand specified? - Don't read the shebang; only rely on the metadata to limit the Python version - Execute what's in `bin/` for the virtual environment if command name is specified? - Fall back to `-m` if nothing found in `bin/` matching the name (and it wasn't a valid path)? ### `py list` - [`hatch python show`](https://hatch.pypa.io/latest/cli/reference/#hatch-python-show) - [`pdm python list`](https://pdm-project.org/en/latest/reference/cli/#list_2) - [`uv python list`](https://docs.astral.sh/uv/reference/cli/#uv-python-list) - [`pipx interpreter list`](https://pipx.pypa.io/latest/docs/#pipx-interpreter) - [`pymanager list`](https://peps.python.org/pep-0773/#list-subcommand) List all of the discovered interpreters. - [Discourse discussion](https://discuss.python.org/t/what-information-is-useful-to-know-statically-about-an-interpreter/25563) - [PEP 711 cross-post](https://discuss.python.org/t/pep-711-pybi-a-standard-format-for-distributing-python-binaries/25547/14?u=brettcannon) - `--format`: Specify the output format; all output in UTF-8 - `jsonl`: Print [interpreter details](https://github.com/brettcannon/python-launcher/discussions/168) using [JSON Lines](https://jsonlines.org/) - This should work with [`json -o` for Nu](https://www.nushell.sh/commands/docs/from_json.html) - Results are streamed as results are found instead of being sorted - Fields - `path` - `virtual` - `found via` - `$VIRTUAL_ENV` - `.venv` - `$PATH` - `pretty` (default): Print interpreter details in a format designed to be human-readable - Include same details as the JSONL output - Sorted by how "good" the interpreter is - `--ignore-venv`: Ignore virtual environments (or default to `--include-venv`) - Might be unnecessary or only useful via `--one` - `--one` - Only print the path to the "best" interpreter - https://github.com/python/cpython/issues/102522 would give more accurate version info for virtual environments ### `py venv` - [`pdm venv`](https://pdm-project.org/latest/reference/cli/#venv) Requires [[Virtual environment location]] landing. - Create a virtual environment in `.venv` next to `pyproject.toml` (this should be the same as what `py run` would do if it was pointed at a project to create a virtual environment) - Without pip, but that might require a `py pip` command or some other abstraction over installation of dependencies - Provide a way to get the location of the virtual environment, else error out? - Provide a way to point to a preexisting or already active virtual environment? - Provide the command to source to activate the virtual environment? ### `py completions` - [`pdm completion`](https://pdm-project.org/latest/reference/cli/#completion) Print the completion script for the specified shell. It would be a command instead of a static file so that it can be accurate for what interpreters are installed. But if the shells provide a way to do it dynamically then this might not be worth it. ### `py tool` - [`pipx run`](https://pipx.pypa.io/latest/docs/#pipx-run) - [`uv tool`](https://docs.astral.sh/uv/reference/cli/#uv-tool) Install and/or run tools from e.g. PyPI. Trickiest bit is the UI for installing versus running such that running is very short when you don't want to bother installing (`py x`?). ### `py sync` - [`pdm sync`](https://pdm-project.org/latest/reference/cli/#sync) - [`uv sync`](https://docs.astral.sh/uv/reference/cli/#uv-sync) - [`poetry sync`](https://python-poetry.org/docs/cli/#sync) - [Hatch syncs the environments automatically](https://hatch.pypa.io/latest/environment/#dependencies) whenever you run a command Create/guarantee the virtual environment has the appropriate packages installed. ### `py python` - [pymanager install](https://peps.python.org/pep-0773/#install-subcommand) - [`uv python`](https://docs.astral.sh/uv/reference/cli/#uv-python) - [`poetry python`](https://python-poetry.org/docs/cli/#python) - [`hatch python`](https://hatch.pypa.io/latest/cli/reference/#hatch-python) - [`pdm python`](https://pdm-project.org/latest/reference/cli/#python) ([`pdm py`](https://pdm-project.org/latest/reference/cli/#py)) Install/manage Python interpreters. - Worry about anything other than CPython? - PyPy? - Any builds other than the native CPU? - WASI? ## Library crates Wait until after any `py install` command happens and seeing if relocatable builds could be used to create a Python Launcher written in Python. - Create `locatepython` crate (or `findpython`), based around JSON schema - Add Windows support - Share across `py`, PVSC, and `uv` - Shebang reading? - Inline metadata parsing? - `requires-python` handling? - Cached pip handling? ## Windows support - [Virtual environments](https://github.com/brettcannon/python-launcher/issues/57) - [`PATH`](https://github.com/brettcannon/python-launcher/issues/14) - [Registry](https://github.com/brettcannon/python-launcher/issues/15) - [Microsoft Store](https://github.com/brettcannon/python-launcher/issues/16)