In information theory, there is a concept called an oracle[1] Not to be confused with Larry Ellison's law firm posing as a software company. - a system that can reliably and correctly answer a question of a given kind[2] en.wikipedia.org/wiki/Oracle_machine. When the term is used in software, it is misappropriated to describe a machine whose behavior is authoritative[3] For example, during a migration from System A (the original, trusted system) to System B (the faster, modern replacement), engineers will treat System A like an oracle, and try to cajole or brute-force system B into lockstep..
Thus when we speak of an oracle in the world of software we're often really thinking about very large sets of pairs - inputs and outputs. ML, AI, reverse engineering, cryptography - all of these fields interrogate, predict, and emulate system behavior through analyzing pairs of stimulus and response.
The Oracle from antiquity, the one at the temple of Apollo at Delphi, profoundly changed the course of Greek history. Many of her prophecies not just foretold the fates of city states and empires, but influenced the choices that developed and doomed them. These include Athens' admonishment to "trust in wooden walls", Phillip II's hint toward "silver spears" to conquer the world, and Croesus' "destruction of a mighty empire if you attack" (it turned out to be his own)[4]en.wikipedia.org/wiki/List_of_oracular_statements_from_Delphi[5]Outside of conquest, the Delphic Oracle also counseled Cicero to find his voice, set in motion Nero's fate, and inspired the Spartan code of morality.. When believed, prophecy changes the course of events through its utterance.
Through a plethora of inputs and outputs, I've gained both appreciation for and skepticism of AI[6]Like most other Software Engineers, I have found my work with AI transfixing - in a very short time period, everything is suddenly possible, malleable, concerning, and surprising.. Some capabilities - reasoning, discovery, creativity - are developing quickly and already proving useful. Some flaws - accuracy, judgement, and empathy to name a few - seem like they are stuck, unlikely to be improved by a bigger training run. Many of the things that our minds are capable of are cultivated through experience, internal state, relationship, and the divine.
The Oracle at Delphi had a title - that of Pythia (a role assumed by many individuals over time, like the Pope)[7]en.wikipedia.org/wiki/Pythia. Pythia - the high Priestess - was the woman charged with dispensing Apollo's wisdom. While accounts of how she worked are scarce and conflicting, this may have involved the inhalation of psychedelic fumes from the earth, the transcription and interpretation of other priestesses and priests, and a variety of cleansing rituals to prepare both the Oracle and supplicant for the truth.
Pythia served as the conduit for something - whether it was fumes, the informational networks of the well informed clergy[8]It is commonly thought that Pythia was aided by the large staff of the temple of Apollo, whose foreign religious emissaries made the temple of Apollo a hub of information about the Greek world, and might slightly demystify some of the more prescient prophecies., or the will of Apollo - she was the pipe through which humans conversed with something else, and through that counsel charted the course of human events.
LLMs have intelligence of a different kind. Unlocking their value in contexts that matter to us humans will require deterministic processes that are attuned to their failure modes and peculiarities - software that aims to be a conduit between this alien intelligence, and our human preferences, values, and ambitions.
This is the vision that animates Pythia Software: to build translation between mechanistic capabilities, and human goals.
Our first two projects build infrastructure around LLMs to produce reliability and judgement that their stochastic nature cannot achieve by default.
* In the first, we're using swarms of agents to automate black-box migrations, with Excel as a first target. Excel is the lingua franca of computational modelers in actuary, finance, and the environment, but is often too slow for simulation. Our solution - xplo has used extensive LLM scaffolding to painstakingly reproduce the Excel engine. That's just the start - this infrastructure will eventually generate replacements for deterministic systems of all kinds.
* In the second, we're building software to quantify text against arbitrary human described criteria. LLMs are famously bad at generating numerical grades, but are competent at giving qualitative feedback and answering straightforward questions. Our project, Tally, aims to explore how we can elicit numerical evaluation from LLMs that comports with human judgement, even though these systems think in terms of words, not numbers. Tally is showing early promise in solving problems like scientific reputation, content alignment, and subjective filtration.
AI is young - how it will be scaffolded, factored, and interpreted is still unsettled, and could be profoundly harmful. Technologists have a bad habit of allowing the quirks and contours of technology to drive how it is applied, rather than following and serving the needs of people. Pythia will be different - a conduit through which humans can ask qustions and recieve answers.