Best Form Arguments
A Firm Foundation for Experimental Policy
A Firm Foundation for Policy
A previous piece made the argument that morality is not a good foundation for policy making, legislation, and collaborative decision making.
Here we’ll attempt to invert the question and define a solid foundation for collaboratively designing and crafting experimental policy and evaluating it’s impact.
We’ve already talked about the “why” for this approach.
This piece should be considered the “what”. The next piece will start to provide the “how” and propose a software solution to facilitate the development and evaluation of experimental policy.
Before we do that though, let’s start with the obvious, crafting policy is an incredibly complex endeavor. The world is mind-boggling complex, interconnected, and stochastic in nature.
I’ve talked before about how one of the best ways to improve law and policy is to treat every law and policy explicitly as a testable hypothesis.
But just to reiterate every law represents a series of implicit, or hopefully explicit, hypothesis tests: “If we implement this law, it will have an effect”. But the question is not as simple as that because any experimental policy immediately explodes into a panoply of linked questions as soon as we start to discuss its efficacy.
What are the explicit effects of this policy? A 3% tax on all diesel fuel sold in California.
What are the implicit effects of this policy? Possible revamping of vehicles to use other fuels, retrofitting existing vehicles for fuel efficiency, replacement of ground-based diesel vehicles with other transport options (rail, air) for delivery, etc etc.
What are the immediate benefits of this policy? Increased tax revenue.
What are the immediate costs of this policy? Increased costs to consumers of diesel fuels.
How will the effects of the policy evolve over time? Change in mix of vehicle fleet, decrease in use of diesel fuel, possible impact on transportation and infrastructure.
What are the future benefits of this policy? Reduced emissions, cleaner vehicle fleet,
What are the future costs of this policy? Lack of competitiveness for businesses reliant on diesel fuel in California. Movement of these businesses to other locales
What interactions will this policy have with other policies of this jurisdiction? Increase in diesel consumption in other states and at state border to avoid tax. Increase in transportation of bulk diesel fuel into California.
What externalities does this policy introduce? Could shift transport to even more polluting fuels. Could reduce competitiveness. Could create bottleneck in electrical vehicle production.
What externalities will affect the benefits/costs of this policy? Future developments in engine technologies that could make diesel fuel less polluting. New energy options. Changes in transportation mixture.
There is also one other big consideration when evaluating a policy:
What alternative policies are competing with this policy? A n% tax on all diesel fuel sold in California. A moratorium on the production of diesel vehicles. A ban on diesel vehicles. A levey on each new diesel vehicle produced.
Crafting policy with a view on holistic net good can involve navigating numerous cross-functional domains and weighing up a litany of costs and benefits with limitations on both the current data available today and the always unstable conditions of the future— and then comparing all of these against alternative policy proposals.
The point I want to make here is that some of this complexity is irreducible.
To grudgingly quote Karl Rove:
there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know.
As with any scientific or statistical endeavor and with a good Hume-ian foundation we know that we can get better and better evidence to support a hypothesis, pedantically, reject the null hypothesis, over time with the understanding that we are building upon the unstable foundation of habit and custom but can nonetheless improve the quality of our evidence and evaluation of current and potential future policy over time.
An added benefit is that we can also get better at forecasting and understanding the type, degree, and direction of our estimation errors. We not only get better over time at creating and designing policy we also get better at forecasting the impact and uncertainty of these policy changes.
To continue to beat the drum, when we explicitly treat the changes we make to our society as experiments we make better policies, make smarter pivots, generate new hypotheses, gather better and more expansive evidence, and improve experimental design for future changes.
Robust Experimental Policy
Our goal is to develop a robust basis for experimental policy. We want policies that are able to withstand rigorous testing and evaluation yet be flexible enough to change and adapt on the basis of new information, and be broad enough to encompass the holistic nature of empirical reality. In line with treating law as hypothesis testing, the most stable foundation for policy position is the “best form of the argument” (BFA) for that policy. But let’s unpack what exactly the characteristics of a BFA are.
I’d argue the best form of a policy position has the following characteristics:
Empirical: We throw this out a lot but just as we discard morality in our policy prescriptions we also discard anything else not grounded in our empirical reality. Note that we are not saying quantitative here, the distinction is subtle but art museums may well have an empirical value even if that value is difficult to quantify.
Pragmatic: We are interested in policy as a tool to predict and shape outcomes and measure actual results against forecasts. We want practical policy, quality data, and clear analysis.
Universal values: The values underpinning our policy are humanistic and universal. This is Engineering Open Societies, we’re interested in open societies, human flourishing, and again pragmatic improvements to the human condition.
Localizable implementation: While the values that gird are policy are universal part of being pragmatic is understanding that the best, most pragmatic policy in a given jurisdiction will be localized to account for what is achievable in the existing local context, the people, existing laws, current politics, culture, etc. We are pragmatists not idealists, measurable improvement against the status quo is always our goal and the best form of the argument for a given jurisdiction must take into account the current state of that jurisdiction.
Multilevel Explainability: The BFA is not a single form. The strength of a scientific principle, like the law of gravity, lies not only in its evidence but in its adaptability across different levels of understanding. A concept that holds true across contexts remains valid whether it’s simplified for a preschooler or rigorously detailed for a physicist. Drawing on the ideas of Richard Feynman, our BFA means a policy can be understood at multiple layers without losing its essence. The validity of our policy should not change but it should adaptable to diverse levels of understanding and perspectives without losing its robustness and humanism. Multilevel explainability is key in making the policy intelligible to to the population at large and making it a key resource in the democratic political processes of open societies.
Exhaustive: The BFA addresses all of the available evidence on the topic and seeks for historical and comparative situations and experiments. This does not mean that all of this other evidence is accepted or treated equally only that wherever possible it is considered and addressed.
Evolving & Long-term: Our BFA must evolve over time in response to new evidence, changing conditions, additional input etc. The BFA is never static. It must always adapt both to the present condition, current evidence, and the probabilistic distribution of future outcomes. Habit and custom.
Diverse, Guided, Expert Collaboration: Going back to our scientific comparison and our reference to externalities. No policy exists in a vacuum. Cross-functional experts must be able to engage with each other over policy that touches on their areas of expertise. Best form arguments can only arise from collaboration between and with reference to diverse backgrounds and perspectives. This interaction must be structured so as to encourage collaborative, holistic BFA than competing splintered narrow-interest arguments. Our approach is integrative in nature.
Composable: Policies should be composable into larger syntheses of positions that evaluate the effect of policies in conjunction with each other and the current state of play, just as our knowledge of our physical reality is composed of numerous overlapping physical theories and laws that apply simultaneously to our material universe. No policy exists in a silo untouched by the effects of other policies and empirical reality. Policy for larger jurisdictions should be composable from smaller member jurisdictions and individual jurisdictional policy should adhere to policy from larger jurisdictions and ultimately universal values. We want to encourage harmony, or at least consistency, across different jurisdictional levels.
Parsimonious: As always when crafting policies we want to include only that which adds value this means avoiding jargon, moral arguments, and everything else that takes away from the BFA.
The Dream
At some point in the past, perhaps, there may have been polymaths who could engage in government while at the same time having full command of the material conditions of the societies they lived in. This is not possible today, if it ever was, the world is simply too complex, interconnected, and mutable. However, our world has more to offer than ever because of differentiation and specialization. What is proposed here is exactly this. The ability for diverse specialists to build complex “supply chains” to develop and analyze policy resulting in high-quality policy “goods”. These “goods” can then be consumed and provide value to the population at large.
The dream here is that at some point politics becomes something similar to merge conflict resolution à la Git. Where the work of politicians becomes simply cherry-picking existing policy proposals into legislation for their jurisdiction or merging and resolving conflicts with overlapping proposals for their jurisdiction.
Or, viewed another way, the role of government is similar to maintaining a set of linked dependencies for a project à la package management, npm where the role of an individual government is to maintain and update their policies based on the latest best practices and resolve any resulting errors in the resulting governing framework “code”.
As a software engineer, my brain has been conditioned to send up alarm bells whenever I hear the words just and should e.g. “We should just sync our internal collection with what’s provided by the 3rd party API”. These words nearly always stand in the way of hidden complexity or effort.
The same is nearly (although not always) true with government any sentence that takes the form or has the implication “the government can/should/must just do X” must be treated skeptically.
That said, I remain committed to the idea that a repository of experimental policy is the foundation on which to build iteratively improving governments. I look forward to exploring how we can achieve this in a following post.