[{"data":1,"prerenderedAt":471},["ShallowReactive",2],{"NoscriptNav_XrRK2e2e8meJ0jKVGkb5ULGQDVi3UiFQ9nupAr7Yns":3,"\u002Freports\u002Fbinary-dependencies-identifying-the-hidden-packages-we-all-depend-on":8},["Island",4],{"key":5,"result":6},"NoscriptNav_XrRK2e2e8meJ0jKVGkb5ULGQDVi3UiFQ9nupAr7Yns",{"head":7},{},{"id":9,"title":10,"authors":11,"body":13,"canonicalUrl":456,"canonicalWebsiteName":457,"category":458,"date":459,"description":460,"extension":461,"featured":23,"fullWidthLayout":462,"image":463,"imageAlt":463,"location":464,"meta":465,"metaImage":466,"navigation":23,"path":467,"seo":468,"stem":469,"venue":55,"venueUrl":52,"__hash__":470},"reports\u002Freports\u002Fbinary-dependencies-identifying-the-hidden-packages-we-all-depend-on.md","Binary Dependencies: Identifying the Hidden Packages We All Depend On",[12],"vlad",{"type":14,"value":15,"toc":450},"minimark",[16,46,62,65,82,87,98,114,121,124,178,181,184,187,195,199],[17,18,19,20,19,38],"figure",{},"\n    ",[21,22,26,27,32,33,37],"video",{"controls":23,"width":24,"poster":25},true,980,"https:\u002F\u002Fvlad.website\u002Fstatic\u002Fvlad-fosdem-2026.webp","\n        ",[28,29],"source",{"type":30,"src":31},"video\u002Fmp4","https:\u002F\u002Fvlad.website\u002Fstatic\u002Fvlad-fosdem-2026.mp4","\n        Download the ",[34,35,36],"a",{"href":31},"mp4 video",".\n    ",[39,40,41,42,19],"figcaption",{},"\n        Or watch on ",[34,43,45],{"href":44},"https:\u002F\u002Fwww.youtube.com\u002Fwatch?v=fP8OFI5PdBI","YouTube",[47,48,49,50,56,57,61],"p",{},"On 31 Jan 2026, I gave a talk at ",[34,51,55],{"href":52,"rel":53},"https:\u002F\u002Ffosdem.org\u002F2026\u002F",[54],"nofollow","FOSDEM 2026"," on ",[58,59,60],"em",{},"phantom binary dependencies"," — packages that we depend on in\nbinary form, even though these dependency relationships are invisible to us. If we cannot reliably identify these\nphantom dependencies, the sustainability and security of our tech infrastructure will be at risk, which threatens\ncritical services such as hospitals, transportation and the internet.",[47,63,64],{},"You can watch my talk on this page, and I've included more details below, as well as a list of resources for those who\nwant to learn more about this topic.",[66,67,19,70,19,77],"div",{"className":68},[69],"page-items",[34,71,76],{"className":72,"href":75},[73,74],"button","button--arrow","https:\u002F\u002Ffosdem.org\u002F2026\u002Fschedule\u002Fevent\u002F7NQJNU-binary_dependencies_identifying_the_hidden_packages_we_all_depend_on\u002F","\n        See talk info\n    ",[34,78,81],{"className":79,"href":80},[73,74],"https:\u002F\u002Fvlad.website\u002Fstatic\u002Fvlad-fosdem-2026.pdf","\n        See slides\n    ",[83,84,86],"h2",{"id":85},"abstract","Abstract",[47,88,89,90,93,94,97],{},"When you create a software package, your work might depend on other packages. Usually, you will depend on the ",[58,91,92],{},"source\ncode"," of these other packages. However, sometimes, you will depend on ",[58,95,96],{},"precompiled binaries"," of your dependencies. This\nfrequently happens when calling compiled code, like C code, from other programming languages, such as Python.",[47,99,100,101,105,106,109,110,113],{},"In almost all ecosystems, it is difficult to keep track of binary dependencies. When you depend on a package's source\ncode, this is normally recorded in your manifest file — ",[102,103,104],"code",{},"pyproject.toml",", ",[102,107,108],{},"package.json"," and so on. However, when you\ndepend on a package's precompiled binaries, this information is usually not recorded anywhere. This means that the\nbinary dependency relationship between your project and whatever you're depending on is hidden — so we can say that you\nhave a ",[58,111,112],{},"phantom binary dependency",".",[47,115,116,117,113],{},"You can find detailed technical information about how binary dependencies work in my article titled\n",[34,118,120],{"href":119},"\u002Freports\u002Fhow-binary-dependencies-work\u002F","How Binary Dependencies Work Across Different Languages",[47,122,123],{},"Why are phantom binary dependencies important? For at least two reasons:",[125,126,127,172],"ul",{},[128,129,130,134,135,141,142,147,148,153,154,159,160,165,166,171],"li",{},[131,132,133],"strong",{},"Sustainability",".\n",[34,136,140],{"href":137,"className":138},"\u002Fideas\u002Fkeystone-maintainers-keep-the-internet-going\u002F",[139],"suplink","Keystone"," maintainers struggle to get paid\nbecause of the\n",[34,143,146],{"href":144,"className":145},"https:\u002F\u002Fopenpath.quest\u002F2024\u002Fthe-open-source-sustainability-crisis\u002F",[139],"Open Source sustainability crisis",",\nwhich makes them more vulnerable to\n",[34,149,152],{"href":150,"className":151},"https:\u002F\u002Fopensourcepledge.com\u002Fblog\u002Fburnout-in-open-source-a-structural-problem-we-can-fix-together\u002F",[139],"\nburnout",".\nProjects like the\n",[34,155,158],{"href":156,"className":157},"https:\u002F\u002Fopensourcepledge.com",[139],"Open Source Pledge",",\nthe\n",[34,161,164],{"href":162,"className":163},"https:\u002F\u002Fendowment.dev",[139],"Open Source Endowment",",\nand\n",[34,167,170],{"href":168,"className":169},"https:\u002F\u002Fthanks.dev",[139],"thanks.dev","\nhelp maintainers get paid. But we have to know which maintainers we depend on to be able to pay them. If we cannot\nidentify our binary dependencies, we cannot identify which maintainers we should support, which puts the\nsustainability of the Open Source ecosystem at risk, threating our global tech infrastructure.",[128,173,174,177],{},[131,175,176],{},"Security",".\nIf your project depends on a library, any security issues in that library will leave your project vulnerable. If you\ndon't have a clear picture of which libraries you depend on, you won't have a clear picture of security\nvulnerabilities that might affect you, which puts your project at risk. For software that supports critical\ninfrastructure like hospitals, transportation systems, and the internet, this vulnerability puts the public at risk of\nharm.",[47,179,180],{},"There are tools we can build, and systemic changes we can make to our package management ecosystems, that will correct\nthese issues, though this will be a lot of work. In my talk, I sketch the beginnings of a solution.",[47,182,183],{},"I believe we should start by creating tools that can identify and record binary dependencies for a wide variety of\npackages, giving maintainers, security researchers, and other parties the information they need to figure out more\nrobust solutions.",[47,185,186],{},"Once we have this information, we can work on other aspects of the problem. How can we made sure that binary dependencies\nare always sourced from the appropriate package managers, instead of being merely vendored, so that they can be kept up\nto date with security patches? How can we make sure that language package managers and system package managers\ninteroperate to warn developers of security issues across the entire dependency graph?",[47,188,189,190],{},"I'm actively involved in this work, and there are many conversations happening around these questions. If you're\ninterested, here are some resources to help you learn more and\u002For get involved! ",[191,192],"span",{"className":193},[194],"ender",[83,196,198],{"id":197},"other-resources","Other Resources",[125,200,201,210,216,226,240,250,263,277,297,308,316,334,344,354,369,400,421,437],{},[128,202,203,204,209],{},"The ",[34,205,208],{"href":206,"rel":207},"https:\u002F\u002Ftangled.org\u002Fvlad.website\u002Fbindep",[54],"bindep repository"," is where I keep my work-in-progress code and notes.",[128,211,203,212,215],{},[34,213,158],{"href":156,"rel":214},[54]," is an initiative aiming to get companies to pay the Open Source maintainers whose work\nthey depend on. Pledge members have paid $6,879,498 to Open Source developers at the time of writing.",[128,217,218,225],{},[34,219,222],{"href":220,"rel":221},"https:\u002F\u002Fnesbitt.io\u002F2026\u002F01\u002F27\u002Fthe-c-shaped-hole-in-package-management.html",[54],[58,223,224],{},"The C-Shaped Hole in Package Management"," by Andrew Nesbitt, a great post about the problem of binary\ndependencies.",[128,227,228,235,236,239],{},[34,229,232],{"href":230,"rel":231},"https:\u002F\u002Fwww.endorlabs.com\u002Flearn\u002Fdependency-resolution-in-python-beware-the-phantom-dependency",[54],[58,233,234],{},"Dependency Resolution in Python: Beware The Phantom Dependency"," by Anand Sawant is where the term ",[58,237,238],{},"phantom\ndependency"," was originally coined.",[128,241,242,249],{},[34,243,246],{"href":244,"rel":245},"https:\u002F\u002Fgithub.com\u002Fpypa\u002Fauditwheel",[54],[58,247,248],{},"auditwheel",", a widely-used Python package that can identify a wheel's required dynamic libraries, but\ndoes not yet have accurate human-readable output, or an API for researchers and developers to use.",[128,251,252,56,257,262],{},[34,253,256],{"href":254,"rel":255},"https:\u002F\u002Fgithub.com\u002Fpypa\u002Fauditwheel\u002Fissues\u002F676",[54],"Issue #676",[34,258,260],{"href":244,"rel":259},[54],[58,261,248],{},", where I ask the maintainers whether they are interested in\nadding APIs that would give users and researchers more visibility into binary dependencies.",[128,264,265,272,273,276],{},[34,266,269],{"href":267,"rel":268},"https:\u002F\u002Fgithub.com\u002Fpython-wheel-build\u002Felfdeps\u002F",[54],[58,270,271],{},"elfdeps",", a lesser-known Python package that can identify a wheel's required dynamic libraries and ",[58,274,275],{},"does\nhave"," a public API.",[128,278,279,284,285,290,291,296],{},[34,280,283],{"href":281,"rel":282},"https:\u002F\u002Fsethmlarson.dev\u002Fpep-770-sbom-data-from-pypi-fedora-and-redhat",[54],"PEP 770",", in which author Seth Larson introduces SBOMs to Python packages. See also his\n",[34,286,289],{"href":287,"rel":288},"https:\u002F\u002Fgithub.com\u002Fpypa\u002Fauditwheel\u002Fpull\u002F577",[54],"PR #577"," to ",[34,292,294],{"href":244,"rel":293},[54],[58,295,248],{},", which enables users to discover binary dependencies,\nand include them in a package's SBOM.",[128,298,299,304,305,307],{},[34,300,303],{"href":301,"rel":302},"https:\u002F\u002Fpeps.python.org\u002Fpep-0725\u002F",[54],"PEP 725",", which specifies a way to record binary dependencies in ",[102,306,104],{},". This interoperable record\nof binary dependencies could allow many tools to, for example, flag security issues that were previously invisible.",[128,309,310,315],{},[34,311,314],{"href":312,"rel":313},"https:\u002F\u002Fpeps.python.org\u002Fpep-0804\u002F",[54],"PEP 804",", which specifies a system for mapping binary dependency names to packages in non-Python registries.",[128,317,318,323,324,329,330,333],{},[34,319,322],{"href":320,"rel":321},"https:\u002F\u002Fgithub.com\u002Fecosyste-ms\u002Fpackages\u002Fissues\u002F1261",[54],"Issue #1261"," on the ",[34,325,328],{"href":326,"className":327},"https:\u002F\u002Fecosyste.ms",[139],"ecosyste.ms"," ",[58,331,332],{},"packages","\nrepo collects more information on strategies for tracking binary dependencies across multiple package managers.",[128,335,336,343],{},[34,337,340],{"href":338,"rel":339},"https:\u002F\u002Fgithub.com\u002Fsony\u002Fesstra",[54],[58,341,342],{},"ESSTRA"," is a tool developed at Sony that aims to improve supply chain transparency by embedding metadata\ninto binaries.",[128,345,346,353],{},[34,347,350],{"href":348,"rel":349},"https:\u002F\u002Fgithub.com\u002Fpython-wheel-build\u002Ffromager",[54],[58,351,352],{},"Fromager"," is a tool that aims, among other things, to provide a way for Python packages to be built\ncompletely from source, which includes building all their binary dependencies from source. I am not sure whether this\nis actually implemented yet.",[128,355,356,361,362,113],{},[34,357,360],{"href":358,"rel":359},"https:\u002F\u002Fuapi-group.org\u002Fspecifications\u002Fspecs\u002Fpackage_metadata_for_executable_files\u002F",[54],"UAPI specification 8"," provides a section within ELF binaries for recording the provenance of a dynamic\nlibrary, including the name of the system package it originated from. See also Fedora's page about ",[34,363,366],{"href":364,"rel":365},"https:\u002F\u002Ffedoraproject.org\u002Fwiki\u002FChanges\u002FPackage_information_on_ELF_objects",[54],[58,367,368],{},"Package\ninformation on ELF objects",[128,370,371,376,377,380,381,388,389,396,397,399],{},[34,372,375],{"href":373,"rel":374},"https:\u002F\u002Fuapi-group.org\u002Fspecifications\u002Fspecs\u002Felf_dlopen_metadata\u002F",[54],"UAPI specification 12"," provides a section within ELF binaries for recording the names of libraries loaded\nwith ",[102,378,379],{},"dlopen()",". This would hypothetically allow us to keep track of libraries opened with\n",[34,382,385],{"href":383,"rel":384},"https:\u002F\u002Fgithub.com\u002Flibffi\u002Flibffi",[54],[58,386,387],{},"libffi","\u002F",[34,390,393],{"href":391,"rel":392},"https:\u002F\u002Fcffi.readthedocs.io\u002Fen\u002Fstable\u002F",[54],[58,394,395],{},"cffi",". However, I'm not sure how these records would be filled in, since it's possible for\nthe names of libraries opened with ",[102,398,379],{}," to not be known until runtime.",[128,401,402,403,410,411,416,417,420],{},"In a paper titled ",[34,404,407],{"href":405,"rel":406},"https:\u002F\u002Fdl.acm.org\u002Fdoi\u002Fpdf\u002F10.1145\u002F3551349.3556921",[54],[58,408,409],{},"Insight: Exploring Cross-Ecosystem Vulnerability Impacts",", Xu et al describe a system\nfor identifying when Python code calls into parts of C-based dynamic libraries that are known to have security\nvulnerabilities. According to their research, 24.0% of PyPI projects “transitively invoke vulnerable APIs from C\nlibraries”. Note, however, that Xu et al analyse Python code that calls into binary dependencies ",[58,412,413,414],{},"using ",[102,415,379],{},",\nwhich is a notable caveat, since these kinds of FFI calls are not the most common way to call into binary\ndependencies. See my ",[34,418,419],{"href":119},"post on how binary dependencies work"," for more information.",[128,422,203,423,430,431,436],{},[34,424,427],{"href":425,"rel":426},"https:\u002F\u002Fgithub.com\u002Fpackage-url\u002Fpurl-registry",[54],[58,428,429],{},"PURL registry"," is a registry of packages, mainly C\u002FC++ packages, that do not otherwise have a\n",[34,432,435],{"href":433,"rel":434},"https:\u002F\u002Fgithub.com\u002Fpackage-url\u002Fpurl-spec",[54],"PURL"," because they are not clearly identified in package registries. Such a registry could be used for\nassigning reliable identities to binary dependencies.",[128,438,439,446,447,113],{},[34,440,443],{"href":441,"rel":442},"https:\u002F\u002Fsentry.engineering\u002Fblog\u002Fpublishing-binaries-on-npm",[54],[58,444,445],{},"How to publish binaries on npm"," by Luca Forstner tells us that it's possible, but tricky, to publish\nbinary packages on ",[58,448,449],{},"npm",{"title":451,"searchDepth":452,"depth":452,"links":453},"",2,[454,455],{"id":85,"depth":452,"text":86},{"id":197,"depth":452,"text":198},"https:\u002F\u002Fvlad.website\u002Fbinary-dependencies-identifying-the-hidden-packages-we-all-depend-on\u002F","vlad.website","software-supply-chains","2026-01-31","We need better tools for uncovering phantom binary dependencies. Not having these tools makes our global tech infrastructure less secure, and puts a strain on the Open Source maintainers we rely on.","md",false,null,"Brussels, Belgium",{},"https:\u002F\u002Fvlad.website\u002Fimages\u002Fopengraph\u002Fbinary-dependencies.png","\u002Freports\u002Fbinary-dependencies-identifying-the-hidden-packages-we-all-depend-on",{"title":10,"description":460},"reports\u002Fbinary-dependencies-identifying-the-hidden-packages-we-all-depend-on","aQtO52x-sFx3eV1gThZMBglihmXS6U9v7Ap76loqYrg",1780596102648]