Plans - Open Source by Brett Cannon

# Important links - [`os-wasi` label](https://github.com/python/cpython/labels/OS-wasi) - [WASI dev container definition](https://github.com/python/cpython-devcontainers/tree/main/wasicontainer) ([image](https://github.com/python/cpython-devcontainers/pkgs/container/wasicontainer)) - Buildbot [[builders](https://buildbot.python.org/#/builders?tags=%2Bwasi)ers](https://buildbot.python.org/#/builders?tags=%2Bwasi) # Legend - 🥧 beginner-friendly ("as easy as pie") - 🔖 Working on it (bookmark where I/someone left off) - 🛑 Blocked by something (stop and check status before proceeding) # Plans - [ ] [Write down support](https://peps.python.org/pep-0816/#:~:text=The%20WASI%20version%20and%20WASI%20SDK%20version%20supported%20for%20a%20Python%20version%20MUST%20be%20recorded%20in%20PEP%2011%20as%20an%20official%20record.) for Python 3.15 as outlined in [PEP 816](https://peps.python.org/pep-0816/) - [ ] 🛑 [Support `wasm32-wasip3`](https://github.com/python/cpython/issues/121634) (skipping p2; blocked on WASI 0.3 being released) - [x] Ability to specify host triple - [x] Find out if `wasip2` is compatible w/ preview1 hosts - [ ] buildbot - [ ] PEP 11 - [ ] PEP for a wheel tag ([discussion](https://bytecodealliance.zulipchat.com/#narrow/channel/219900-wasi/topic/Platform.20tags.20for.20packages.20targeting.20WASI/with/516694780)) - [ ] Expose WASI details in `sys` or `platform` ([changes in wasi-libc](https://github.com/WebAssembly/wasi-libc/issues/688)) - [ ] Draft PEP (thanks to PEP 816, might be able to do just `wasi_wasm32` and non-standard `wasi_N_N_sdk_N_wasm32` in `packaging` when not matching PEP 11) - [ ] Post to DPO - [ ] Submit for approval - [ ] `wasi package` - [x] Figure out file name (`python-3.25.0a3-wasm32-wasip1.tar.xz`; host triple from `config.toml`; [`platform.python_version()`](https://docs.python.org/3/library/platform.html#platform.python_version)) - [x] Research what should be shipped for building extensions - `--without-static-libpython` - Top-level stuff in `cross-build/wasm32-wasip1` accounted for (e.g. [`pyconfig.h`](https://github.com/python/cpython/blob/957f9fe162398fceeaa9ddba8b40046b8a03176d/Makefile.pre.in#L2964)) - [`build/lib.*` to the stdlib (sans `__pycache__`)](https://github.com/python/cpython/blob/957f9fe162398fceeaa9ddba8b40046b8a03176d/Makefile.pre.in#L2853-L2855) - [`LICENSE` to `LICENSE.txt`](https://github.com/python/cpython/blob/957f9fe162398fceeaa9ddba8b40046b8a03176d/Makefile.pre.in#L2856) - `include/pythonN.Nd` if debug, otherwise `lib/pythonN.N` - `python.wasm` renamed to `python3.NN.wasm` (although where to put it since it isn't a self-contained binary?) - Try to be generic so it works for any Python build so it can be used to compare against `make install` as well as be used for relocatable builds - [X] [Refactor](https://github.com/brettcannon/cpython/tree/wasi-refactor) `Platforms/WASI` to support other subcommands ([PR](https://github.com/python/cpython/pull/145404)) - [ ] Implement - [ ] All defaults - [ ] Specify the directory/file path - [ ] Specify build directory - [ ] Flag to allow for building if not done, else error out - [ ] Make [reproducible](https://sethmlarson.dev/security-developer-in-residence-weekly-report-14) - [ ] Provide some test script to compare what gets packaged against an install to some directory - [ ] Use the WASI dev container to [build and export the package](https://docs.docker.com/build/building/export/) - [ ] Have the RM release the binaries - [ ] [External dependencies](https://github.com/python/cpython-source-deps) via `wasi externals` - [ ] Figure out what version policy we want (i.e. always latest or freeze versions like Windows builds? If latter, make it an external config) - [ ] [zlib](https://github.com/python/cpython/issues/91246) (and potentially undoing part of https://github.com/python/cpython/pull/130297)/zlib-ng - `RANLIB=/opt/wasi-sdk-29.0-x86_64-linux/bin/ranlib CC=/opt/wasi-sdk-29.0-x86_64-linux/bin/clang CHOST=wasm32 ./configure --static --zlib-compat --prefix ... && make install` - Blank out `PKG_CONFIG_SYSROOT_DIR` and `PKG_CONFIG_PATH`, set `PKG_CONFIG_LIBDIR` to where `lib/pkgconfig` ended up from `--prefix` - [ ] bzip2 - [ ] xz - [`get_externals.bat`](https://github.com/python/cpython/blob/5592399313c963c110280a7c98de974889e1d353/PCbuild/get_externals.bat#L61) - [`python.props`](https://github.com/python/cpython/blob/5592399313c963c110280a7c98de974889e1d353/PCbuild/python.props#L73) - [`externals.spdx.json`](https://github.com/python/cpython/blob/5592399313c963c110280a7c98de974889e1d353/Misc/externals.spdx.json#L160) - [ ] sqlite - [ ] zstd - [ ] mpdecimal - [x] OpenSSL (not safe as WebAssembly doesn't have constant time ops) - [ ] 🛑 Networking under WASI 0.3 (blocked on threading in WASI 0.3 being released) - [Joel's hack](https://github.com/dicej/cpython/commit/118e9d8eeabb6345d57295f6cec1616694eacbc2) - Any tests requiring threads probably won't work - See the [headers in wasi-libc](https://github.com/WebAssembly/wasi-libc/tree/main/libc-top-half/musl/include) for what can be turned on - Will probably require tweaking `config.site` - [ ] 🥧 Make sure build details are still relevant - [ ] Check if all the flags set in `configure.ac` are necessary - [ ] Check what `__wasi__` usage can be removed - [x] [Check](https://github.com/python/cpython/issues/115982) that the [site config](https://github.com/python/cpython/blob/main/Tools/wasm/config.site-wasm32-wasi) is up-to-date - [ ] [[HTTP fetch API]] # Packaging notes ```shell ./configure --without-static-libpython --prefix (pwd)/builddir && make -s -j && make install ``` ## Release ```shell ➲ ls 📂 bin 📂 include 📂 lib 📂 share ``` ```shell ➲ ls bin/ 🔖 idle3 🏗 idle3.15 🏗 pip3 🏗 pip3.15 🔖 pydoc3 🏗 pydoc3.15 🔖 python3 🔖 python3-config 🏗 python3.15 🏗 python3.15-config ``` ```shell ➲ ls include/python3.15/ 📄 abstract.h 📄 codecs.h 📄 dictobject.h 📄 floatobject.h 📄 listobject.h 📄 object.h 📄 py_curses.h 📄 pyexpat.h 📄 pymem.h 📄 pythonrun.h 📄 structmember.h 📄 warnings.h 📄 audit.h 📄 compile.h 📄 dynamic_annotations.h 📄 frameobject.h 📄 longobject.h 📄 objimpl.h 📄 pyatomic.h 📄 pyframe.h 📄 pyport.h 📄 pythread.h 📄 structseq.h 📄 weakrefobject.h 📄 bltinmodule.h 📄 complexobject.h 📄 enumobject.h 📄 genericaliasobject.h 📄 marshal.h 📄 opcode.h 📄 pybuffer.h 📄 pyhash.h 📄 pystate.h 📄 pytypedefs.h 📄 sysmodule.h 📄 boolobject.h 📂 cpython 📄 errcode.h 📄 import.h 📄 memoryobject.h 📄 opcode_ids.h 📄 pycapsule.h 📄 pylifecycle.h 📄 pystats.h 📄 rangeobject.h 📄 traceback.h 📄 bytearrayobject.h 📄 critical_section.h 📄 exports.h 📂 internal 📄 methodobject.h 📄 osdefs.h 📄 pyconfig.h 📄 pymacconfig.h 📄 pystrcmp.h 📄 refcount.h 📄 tupleobject.h 📄 bytesobject.h 📄 datetime.h 📄 fileobject.h 📄 intrcheck.h 📄 modsupport.h 📄 osmodule.h 📄 pydtrace.h 📄 pymacro.h 📄 pystrtod.h 📄 setobject.h 📄 typeslots.h 📄 ceval.h 📄 descrobject.h 📄 fileutils.h 📄 iterobject.h 📄 moduleobject.h 📄 patchlevel.h 📄 pyerrors.h 📄 pymath.h 📄 Python.h 📄 sliceobject.h 📄 unicodeobject.h ``` ```shell ➲ ls lib/ 📂 pkgconfig 📂 python3.15 ``` ```shell ➲ ls lib/pkgconfig/ 📄 python-3.15-embed.pc 📄 python-3.15.pc 🔖 python3-embed.pc 🔖 python3.pc ``` ```shell ➲ ls lib/python3.15/ 📄 __future__.py 📄 _pyio.py 📄 calendar.py 📄 dataclasses.py 📄 gettext.py 📄 locale.py 📄 platform.py 📄 sched.py 📄 symtable.py 📄 types.py 📄 __hello__.py 📄 _pylong.py 📄 cmd.py 📄 datetime.py 📄 glob.py 📂 logging 📄 plistlib.py 📄 secrets.py 📂 sysconfig 📄 typing.py 📂 __phello__ 📂 _pyrepl 📄 code.py 📂 dbm 📄 graphlib.py 📄 lzma.py 📄 poplib.py 📄 selectors.py 📄 tabnanny.py 📂 unittest 📂 __pycache__ 📄 _sitebuiltins.py 📄 codecs.py 📄 decimal.py 📄 gzip.py 📄 mailbox.py 📄 posixpath.py 📄 shelve.py 📄 tarfile.py 📂 urllib 📄 _aix_support.py 📄 _strptime.py 📄 codeop.py 📄 difflib.py 📄 hashlib.py 📄 mimetypes.py 📄 pprint.py 📄 shlex.py 📄 tempfile.py 📄 uuid.py 📄 _android_support.py 📄 _sysconfig_vars__linux_x86_64-linux-gnu.json 📂 collections 📄 dis.py 📄 heapq.py 📄 modulefinder.py 📄 profile.py 📄 shutil.py 📂 test 📂 venv 📄 _apple_support.py 📄 _sysconfigdata__linux_x86_64-linux-gnu.py 📄 colorsys.py 📄 doctest.py 📄 hmac.py 📂 multiprocessing 📂 profiling 📄 signal.py 📄 textwrap.py 📄 warnings.py 📄 _ast_unparse.py 📄 _threading_local.py 📄 compileall.py 📂 email 📂 html 📄 netrc.py 📄 pstats.py 📂 site-packages 📄 this.py 📄 wave.py 📄 _collections_abc.py 📄 _weakrefset.py 📂 compression 📂 encodings 📂 http 📄 ntpath.py 📄 pty.py 📄 site.py 📄 threading.py 📄 weakref.py 📄 _colorize.py 📄 abc.py 📂 concurrent 📂 ensurepip 📂 idlelib 📄 nturl2path.py 📄 py_compile.py 📄 smtplib.py 📄 timeit.py 📄 webbrowser.py 📄 _compat_pickle.py 📄 annotationlib.py 📂 config-3.15-x86_64-linux-gnu 📄 enum.py 📄 imaplib.py 📄 numbers.py 📄 pyclbr.py 📄 socket.py 📂 tkinter 📂 wsgiref 📄 _ios_support.py 📄 antigravity.py 📄 configparser.py 📄 filecmp.py 📂 importlib 📄 opcode.py 📄 pydoc.py 📄 socketserver.py 📄 token.py 📂 xml 📄 _markupbase.py 📄 argparse.py 📄 contextlib.py 📄 fileinput.py 📄 inspect.py 📄 operator.py 📂 pydoc_data 📂 sqlite3 📄 tokenize.py 📂 xmlrpc 📄 _missing_stdlib_info.py 📄 ast.py 📄 contextvars.py 📄 fnmatch.py 📄 io.py 📄 optparse.py 📄 queue.py 📄 ssl.py 📂 tomllib 📄 zipapp.py 📄 _opcode_metadata.py 📂 asyncio 📄 copy.py 📄 fractions.py 📄 ipaddress.py 📄 os.py 📄 quopri.py 📄 stat.py 📄 trace.py 📂 zipfile 📄 _osx_support.py 📄 base64.py 📄 copyreg.py 📄 ftplib.py 📂 json 📂 pathlib 📄 random.py 📄 statistics.py 📄 traceback.py 📄 zipimport.py 📄 _py_abc.py 📄 bdb.py 📄 cProfile.py 📄 functools.py 📄 keyword.py 📄 pdb.py 📂 re 📂 string 📄 tracemalloc.py 📂 zoneinfo 📄 _py_warnings.py 📄 bisect.py 📄 csv.py 📄 genericpath.py 📂 lib-dynload 📄 pickle.py 📄 reprlib.py 📄 stringprep.py 📄 tty.py 📄 _pydatetime.py 📄 build-details.json 📂 ctypes 📄 getopt.py 📄 LICENSE.txt 📄 pickletools.py 📄 rlcompleter.py 📄 struct.py 📄 turtle.py 📄 _pydecimal.py 📄 bz2.py 📂 curses 📄 getpass.py 📄 linecache.py 📄 pkgutil.py 📄 runpy.py 📄 subprocess.py 📂 turtledemo ``` ```shell /tmp/cp-install ➲ ls lib/python3.15/lib-dynload/ 🏗 _bisect.cpython-315-x86_64-linux-gnu.so 🏗 _heapq.cpython-315-x86_64-linux-gnu.so 🏗 _sha2.cpython-315-x86_64-linux-gnu.so 🏗 _zoneinfo.cpython-315-x86_64-linux-gnu.so 🏗 zlib.cpython-315-x86_64-linux-gnu.so 🏗 _blake2.cpython-315-x86_64-linux-gnu.so 🏗 _hmac.cpython-315-x86_64-linux-gnu.so 🏗 _sha3.cpython-315-x86_64-linux-gnu.so 🏗 _zstd.cpython-315-x86_64-linux-gnu.so 🏗 _bz2.cpython-315-x86_64-linux-gnu.so 🏗 _interpchannels.cpython-315-x86_64-linux-gnu.so 🏗 _socket.cpython-315-x86_64-linux-gnu.so 🏗 array.cpython-315-x86_64-linux-gnu.so 🏗 _codecs_cn.cpython-315-x86_64-linux-gnu.so 🏗 _interpqueues.cpython-315-x86_64-linux-gnu.so 🏗 _sqlite3.cpython-315-x86_64-linux-gnu.so 🏗 binascii.cpython-315-x86_64-linux-gnu.so 🏗 _codecs_hk.cpython-315-x86_64-linux-gnu.so 🏗 _interpreters.cpython-315-x86_64-linux-gnu.so 🏗 _ssl.cpython-315-x86_64-linux-gnu.so 🏗 cmath.cpython-315-x86_64-linux-gnu.so 🏗 _codecs_iso2022.cpython-315-x86_64-linux-gnu.so 🏗 _json.cpython-315-x86_64-linux-gnu.so 🏗 _statistics.cpython-315-x86_64-linux-gnu.so 🏗 fcntl.cpython-315-x86_64-linux-gnu.so 🏗 _codecs_jp.cpython-315-x86_64-linux-gnu.so 🏗 _lsprof.cpython-315-x86_64-linux-gnu.so 🏗 _struct.cpython-315-x86_64-linux-gnu.so 🏗 grp.cpython-315-x86_64-linux-gnu.so 🏗 _codecs_kr.cpython-315-x86_64-linux-gnu.so 🏗 _lzma.cpython-315-x86_64-linux-gnu.so 🏗 _testbuffer.cpython-315-x86_64-linux-gnu.so 🏗 math.cpython-315-x86_64-linux-gnu.so 🏗 _codecs_tw.cpython-315-x86_64-linux-gnu.so 🏗 _math_integer.cpython-315-x86_64-linux-gnu.so 🏗 _testcapi.cpython-315-x86_64-linux-gnu.so 🏗 mmap.cpython-315-x86_64-linux-gnu.so 🏗 _csv.cpython-315-x86_64-linux-gnu.so 🏗 _md5.cpython-315-x86_64-linux-gnu.so 🏗 _testclinic.cpython-315-x86_64-linux-gnu.so 🏗 pyexpat.cpython-315-x86_64-linux-gnu.so 🏗 _ctypes.cpython-315-x86_64-linux-gnu.so 🏗 _multibytecodec.cpython-315-x86_64-linux-gnu.so 🏗 _testclinic_limited.cpython-315-x86_64-linux-gnu.so 🏗 readline.cpython-315-x86_64-linux-gnu.so 🏗 _ctypes_test.cpython-315-x86_64-linux-gnu.so 🏗 _multiprocessing.cpython-315-x86_64-linux-gnu.so 🏗 _testimportmultiple.cpython-315-x86_64-linux-gnu.so 🏗 resource.cpython-315-x86_64-linux-gnu.so 🏗 _curses.cpython-315-x86_64-linux-gnu.so 🏗 _pickle.cpython-315-x86_64-linux-gnu.so 🏗 _testinternalcapi.cpython-315-x86_64-linux-gnu.so 🏗 select.cpython-315-x86_64-linux-gnu.so 🏗 _curses_panel.cpython-315-x86_64-linux-gnu.so 🏗 _posixshmem.cpython-315-x86_64-linux-gnu.so 🏗 _testlimitedcapi.cpython-315-x86_64-linux-gnu.so 🏗 syslog.cpython-315-x86_64-linux-gnu.so 🏗 _dbm.cpython-315-x86_64-linux-gnu.so 🏗 _posixsubprocess.cpython-315-x86_64-linux-gnu.so 🏗 _testmultiphase.cpython-315-x86_64-linux-gnu.so 🏗 termios.cpython-315-x86_64-linux-gnu.so 🏗 _decimal.cpython-315-x86_64-linux-gnu.so 🏗 _queue.cpython-315-x86_64-linux-gnu.so 🏗 _testsinglephase.cpython-315-x86_64-linux-gnu.so 🏗 unicodedata.cpython-315-x86_64-linux-gnu.so 🏗 _elementtree.cpython-315-x86_64-linux-gnu.so 🏗 _random.cpython-315-x86_64-linux-gnu.so 🏗 _tkinter.cpython-315-x86_64-linux-gnu.so 🏗 xxlimited.cpython-315-x86_64-linux-gnu.so 🏗 _gdbm.cpython-315-x86_64-linux-gnu.so 🏗 _remote_debugging.cpython-315-x86_64-linux-gnu.so 🏗 _uuid.cpython-315-x86_64-linux-gnu.so 🏗 xxlimited_35.cpython-315-x86_64-linux-gnu.so 🏗 _hashlib.cpython-315-x86_64-linux-gnu.so 🏗 _sha1.cpython-315-x86_64-linux-gnu.so 🏗 _xxtestfuzz.cpython-315-x86_64-linux-gnu.so 🏗 xxsubtype.cpython-315-x86_64-linux-gnu.so ``` ```shell ➲ ls share/man/man1/ 🔖 python3.1 📄 python3.15.1 ``` ## Debug Only listing differences with a release build. ```shell ➲ ls bin/ idle3 pip3 pydoc3 python3 python3.15 python3.15d idle3.15 pip3.15 pydoc3.15 python3-config python3.15-config python3.15d-config ``` ```shell ➲ ls include/python3.15d/ abstract.h datetime.h import.h objimpl.h pyerrors.h pystrcmp.h sysmodule.h audit.h descrobject.h internal opcode.h pyexpat.h pystrtod.h traceback.h bltinmodule.h dictobject.h intrcheck.h opcode_ids.h pyframe.h Python.h tupleobject.h boolobject.h dynamic_annotations.h iterobject.h osdefs.h pyhash.h pythonrun.h typeslots.h bytearrayobject.h enumobject.h listobject.h osmodule.h pylifecycle.h pythread.h unicodeobject.h bytesobject.h errcode.h longobject.h patchlevel.h pymacconfig.h pytypedefs.h warnings.h ceval.h exports.h marshal.h py_curses.h pymacro.h rangeobject.h weakrefobject.h codecs.h fileobject.h memoryobject.h pyatomic.h pymath.h refcount.h compile.h fileutils.h methodobject.h pybuffer.h pymem.h setobject.h complexobject.h floatobject.h modsupport.h pycapsule.h pyport.h sliceobject.h cpython frameobject.h moduleobject.h pyconfig.h pystate.h structmember.h critical_section.h genericaliasobject.h object.h pydtrace.h pystats.h structseq.h ``` ``` ```shell ➲ ls lib/pkgconfig/ python-3.15-embed.pc python-3.15.pc python-3.15d-embed.pc python-3.15d.pc python3-embed.pc python3.pc ```