what am I trying to say here
people keep trying to create cross-compatible extensible APIs
err, people want them. businesses don't because they lose their moats. businesses want them until they become big because they want to encroach on others' moat and then make their own.
minecraft modpacks with crossmod play. there was no standard for doing things? were there underlying modpacks that all they did was provide abstractions, "superclasses" for actual modpacks to build upon in such a way that they were cross compatible? Or were authors just hardcoding other modpacks they enjoyed in a some sort of n^2 approach where every modpack had to build interop with every other modpack
unix pipes are nice. the entire original unix ecosystem is nice. an abstraction of "a stream of bytes" is low enough that anything can work with it easily. if you're an application written in any programming language worth it's salt (including assembly, tbf), you can ingest a stream of bytes.
it's important to note that most core unix tools were being developed together, with collaboration and communication between authors to settle on a way to interop them. and there was some organic growth, too
Not only this, but there were some additional conventions adopted by the original authors of core unix tools. Things like newlines, parseable separators (cut, sort), easily substitutable (sed). line caching/buffering semantics. These conventions are simple enough that most programming languges can make use of them (python's input(), fgets(3))
and the ecosystem has excelled with this! Modern tools are being written every day that (mostly) interop as well as the original tooling. the bar is low enough and the API is intuitive enough and the ecosystem kind of forces the API with the tty being the primary interface anyways, a stream of bytes.
ecosystems
powershell does something different. it tries to use structured data as it's primary piping dataformat. I don't know enough about powershell to know if this worked out for them in windows land, but I imagine it's harder to write against.
my understanding of SOAP is that it's an attempt at standardizing datastructure serialization & transmission.
ASN.1 also exists.
MCPs also exist, with the additional spin of having an LLM on top of them that can maybe collate and interop data between disparate sources that haven't explicitly planned to be interoperable, without manually writing code to interop between them
what a lot of programming is: plumbing work
sooo much dev work is taking disparate datasources & libraries that don't intend to interop or compose and, well, compose them. translating between datatypes, interleaving function calls from different libraries, including some business logic as well.
some languages try to provide more of a baseline above a pretty generic "array" type that C provides and a convention of "a string is an array with a null delimiter". Python provides generalized iterators, maps/dicts, strings (both unicode, and not). numbers. languages like go and java go the extra mile and provide a very extensive standard library to build against.
I loooooove go and java for this.
I'm quite proud of my szip project, in part because it's built to be completely interoperable with go stdlib's tar reader interface. what this means is, if you use my project to open a zip file, you can pass the resulting reader object to any function in any project built to accept the standard go tar reader.
java has a similarly extensive standard library with all kinds of whacky datastructures to target. and conversions / utility functions to go between datastructures when that makes sense (things like streams to lists or w/e)
compare this to rust. rust tends to take the stance of "small stdlib, extensible / composable library dependencies". there is a kind of convention in rust to expose your publically usable structs or traits as a sub-crate, or a different crate that are importable without importing the main meat of the library. to enable hard-coded interop between projects if they desire to do so.
until you get versioning involved.
a common problem is when two different libraries use the same datastructure, perhaps from the same imported third project, but they pin their dependencies too strictly and so result in different versions of the datastructures. there's no way to interop these. not in rust, I don't think. There's a theoretical world where datastructure upgrades are optionally provided a graceful upgrade and downgrade path such that small version bumps don't result in this phenomenon happening.
rust's interop culture is very n^2, banking on the fact that a winner library is going to sweep the board and that most libraries will target this "winner library" for the datastructures they need, and that other libraries will coincidentally also use the same winner library to result in easy interop between different libraries' usage. the unfortunate paradox about rust, though, is that it also promotes "small" libraries such that there's less monolithic libraries providing a bunch of standard datastructures that is easy to be the clear winner and provide a bunch of interop. instead, a bunch of small libraries compete with other small libraries, some are clear winners but it's not always so clear. lots of unmaintained, previously de-facto libraries for usecases.
aside, but I think the rust devs should provide a "nonstandard library" of a few blessed packages that they say are something people should use. well, built datastructures that would be easy to find as a subset or superset of what you're trying to do with your common datastructures.
compare this to go's datastructure interop culture, which is n to the number of libraries that exist. they just need to target the closest-match datastructure to what exists in the standard library, satisfy some or all of the same traits as the datastructure, maybe provide some conversion helper functions and you can call it a day. you get automatic interop. BUT you also get the downside of being at the behest of the stdlib, a slow-moving behemoth that is difficult to get additions into, difficult to get changed (because so many people are depending on a stable datastructure format). this is why rust avoids the stdlib, they want these things to move faster.
I think there's a balance to be had.
I dream of a world where I'm given standard enough interfaces that work for what I'm trying to do and I can target them. for whatever I'm trying to do: encryption interfaces, URLs, URIs. audio data. abstract "songs". I want a standard, blessed library for every datatype under the sun.
and I want a way to convert these datatypes between languages.
the mediawiki foundation tries to get close, with mediawiki. defining arbitrary relationships on data. so every object might satisfy a subset of what you're looking for. a town someone was born in. etc, provide other examples.
I don't really know where I'm getting with this. the only reason MCPs have become a thing is because AI's been the fad. there is no world in which businesses want their data to be interoperable with other workflows. all businesses want is for users to manually go through their clickthroughs, go to carts. see the dark patterns, be tempted by the dark patterns.
businesses are financially motivated to have their moat
nerds are personally motivated to remove the moat.
musicassistant tries to generalize out the "music player" abstraction to encompass many kinds: spotify, tidal. subsonic. local files. radio.
linux the kernel is like a decent API layer. "files as everything"
https://news.ycombinator.com/item?id=45473357
wikidata
https://www.wikidata.org/wiki/Special:MyLanguage/Help:About_data
opengraph
data standardization is the holy grail. LLMs make it easier to create data for, and use the data.
[[URI]]
protocol URIs were great, specifying the protocol in the url
[[multiformats]] try to do the same, encode the datatype in the hash itself.
where were we in the interop pendulum? where given an http url, there's no encoded way to know the structure of it.
k