Ethical Social Forecasting

Part 1

I’m writing about the ethics (and maybe the legality, too) of social dynamics and forecasting. Also, behavioral dynamics is a related concept, but I’m not going to actively write about this. Some general guidelines that I paraphrased from Duke:

1) Be honest and humble in the presentation and organization of models and tools
2) Defend models that are honest and humble, as incomplete forecasts provide a better sense of the uncertain future than missing forecasts
3) Be kind

Part 2

I wrote this section about an hour after I went down the rabbit hole at the end of Part 2, but I thought it serves as a good introduction…

But, there are some other concerns that I’m exploring, and for that I’m sharing my notes about this blog post. Instability Forecast ModelsIFMs – are used to determine political and economic (human-caused) disasters, but they more generally predict social behavior. The ethics of treating them as ‘social weather’ forecasting instruments might be problematic. One reason might be the reflexive side of social weather which contrasts to a hurricane continuing to head towards Florida – the natural weather forecast doesn’t change the path it takes. Schrodt appears to be thinking about how social and weather models relate to each other, and poses quite a few ethical, pragmatic, and social questions.

Something to read for later, or now if you (or me) has time: http://eventdata.parusanalytics.com/papers.dir/Schrodt.PRL.2.0.pdf

It’s absolutely a cross between Social and Computational Sciences and is titled PATTERNS, RULES AND LEARNING: COMPUTATIONAL MODELS OF INTERNATIONAL BEHAVIOR (Philip A. Schrodt, University of Kansas)

A fun line from the preface (page ix): “Computer scientists seem to find it more useful than political scientists…”

I started to write about the dilemmas presented on Schrodt’s blog, but then I went down a rabbit hole. I was curious about Schrodt, and so I researched his background. Definitely worth researching him on your (my) own (again if it’s me).

Part 3

The main part of this post presents my synthesis of this blog post – written by Phillip Schrodt.

Dilemmas with treating IFMs as Weather Models:

  1. Refexivity – The reflexive qualities of public models (again and for the last time, like a Weather Forecast) make social forecasting problematic (think how Asimovian predictive failures are described in the Foundation Series). Asimov sets a futuristic galaxy in the Foundation Series where social dynamics are predicted by a Mathematical Sociologist called Hari Seldon. Seldon and Daneel Olivaw, another important character, come to the realization of reflexiveness in predicting Social Dynamics. They decide to keep Psychohistory unknown to most people, mainly so that their models continue working.

    In our current time, it might be possible to overcome this dilemma if the models factored in new information, but I’d say that it would be much harder (but, probably not impossible) to build a reflexively resilient (or self-modifying) computational model – BTW, this idea is one of those that isn’t new. Another possible solution (inspired by Weather Models) is to retrain them more frequently. As computing technology scales to size carbon nanotube computation, we will likely be able to retrain ML models more often.


  2. Crying Wolf – There is a dilemma with creating a crisis when there is none. For example, a model incorrectly predicts the possibility of a crisis. People mobilize to stop a crisis that would have never happened. In the most positive events, there might be no harm at all – simply being over-prepared and wasting resources. In the worst cases, humans risk life, limb, freedoms, etc.

    Mitigating this problem echos the point (2) I mentioned earlier from Duke: Defend models that are honest and humble, as incomplete forecasts provide a better sense of the uncertain future than missing forecasts

    Therefore, it might be better to consider the 2nd dilemma as an imperative to overcome – and an imperative in the Kantian sense. In fact, it might be more prudent to develop a Kantian imperative handling this dilemma directly.


  3. Iteration Velocity – Transparency allows for not only sharing of ideas, but new ideas at interfaces between them. Models are biased, so having multiple competing models provides multiple perspectives. It seems that having more models also causes a convergence. At what point does the outlier model make a better prediction?

    I think point ( 1 ) from above provides some nice advice: Be honest and humble in the presentation and organization of models and tools

    Additionally, this dilemma relates to the notion of Scientific Revolutions as they provide new paradigms for considering reality. Without the sharing of social models, it begins to seem that the rate of iteration on new paradigms slows.


  4. Finding Diamonds in piles of $#@! – Especially with the rise of ‘fake news’, the quality of the data mined from the Internet and the data sources you use requires skepticism in order to be understood. Quality is more important than quality, and the dilemma is knowing which sources of data actually maintain a sense of quality.

    Ethically, there is a responsibility to find the higher quality data – otherwise, the risk is that the model might not be as honest.


  5. Data Source Consistency – Different from the quality (as in trustworthiness) – Consistency in this case is referring to the availability of a sustained dataset. Sustained means data that continues to be present over time, and not lost to history. This idea presented itself in the Foundation Series, too. Hari Sledon sought information about historical events of the galactic empire, and expressed to Dors Venabili about the lack of consistent data. The solution Asimov suggests through Seldon might seem obvious, or it might seem less consistent (depends on who you are). Seldon finds that the political and economic center of the empire, Trantor (which is also the capital), serves as a good model to represent the entire history of humanity.

    Similarly, I think the way to solve this dilemma mirrors the Asimovian solution. When we’re trying to build honest models, we need to acknowledge that we might base them on a microcosm – such as D.C, Moscow, Beijing, or choose another capital city of your choice – so long as that microcosm resembles the social and behavioral dynamics you are interested in. If you work for an organization that doesn’t want you to use data on a world capital, then you might need to find another analogous dataset.


  6. Thinking Hard is Important – False data might seem like an easy concept, until you realize that missing, incomplete, biased, and contradictory data carries information that reveals otherwise hidden qualities of reality. Statistical imputation is the process of filling in missing data, and improved computational methods make this easier to do. Another factor to consider: false positives indicate edge cases in statistical and computational approaches – finding better ways of understanding why the false positive happened allows for improvements in our techniques.

    A common example of this is the discovery of infrared. It’s important to notice a thermometer recording heat beyond the red part of the visible spectrum of light (A false positive?). It’s more important to not reject that observation, for example, because it allows for new discoveries (A new model!). William Herschel originally called them ‘Calorific Rays’ – here’s the history.

    Physical Scientists are really good at thinking hard, and have already learned a their fair share of difficult to learn lessons. It’s a good idea to learn from them, too. (Again, quite a few techniques from weather forecasting are increasingly being applied to other domains)


  7. Social means relating to Society (read: People) – Data included in IFMs (and other social models) is related directly to the lived experiences of people. It’s necessary to provide extra attention to how this data deals with people. In other words, those wizards who work with social data ought to regard people as people instead of as statistics.

    There is a great responsibility in understanding that social models that predict and forecast have the capacity to impact the everyday experiences of another person. In fact, they often do. The main goal of Hari Seldon and Daneel Olivaw was to minimize the loss and catastrophe that the galaxy would experience. If you’re thinking about using data for predictive analytics, you have an ethical responsibility to be thinking about the people you will be impacting.

For the source material on these dilemmas, see Schrodt’s Article – it’s a very fascinating discussion. Now I’ve got a better sense of the ethics supporting this blend of social and computational sciences.

Sources:

Leave a Reply

Your email address will not be published. Required fields are marked *