What the Fox Definitely Didn't Say - Avoiding Big Data FalsehoodsBy Chris Lake September 28, 2021
Near the Turkish city of Şanlıurfa stands Göbekli Tepe, a Neolithic monument and one of the most significant archaeological discoveries to date. Since then, archaeologists have uncovered about 5% of the site; discovering monumental T-shaped stone pillars of up to five meters in height, ritual enclosures, and layers of activity stretching back to the Mesolithic period – more than 12,000 years ago.
In the early part of 2017, a group of engineers from the University of Edinburgh published a paper claiming that the entire monument served as a stone-age astronomical observatory and mythological geoglyph – a feature carved into the landscape to record an asteroid impact which caused ‘The Younger Dryas Event’ – thus providing evidence for a controversial theory known as catastrophism.
As much as the claims themselves garnered attention, so too did their methods, which were announced as being based in data analytics. In simple terms, a freeware program called ‘Stellarium 0.15’ was used to calculate which constellations would have been visible from the site of Göbekli Tepe during the late Mesolithic and early Neolithic. They then mapped this to some of the monuments in one of the enclosures and compared the angle and aspect of each constellation with the corresponding animal carvings within the monument. If, for example, the angle of a carved scorpion could be considered analogous to the Scorpius constellation, they would conclude that the carving represented the constellation.
Once they had enough matches to discount the possibility of coincidence, they concluded that the whole monument must have been an observatory. And given that the monument existed at the time of the Younger Dryas, they began interpreting all the other symbols within the monument on that basis – which led them to a date for the asteroid impact.
Needless to say, the core assumptions of this theory are untenable, which led to the research queries being misdirected such that the whole chain from hypothesis to conclusion was entirely flawed. When the archaeologists – who had been digging the site for more than two decades – sent a polite email pointing out these flaws they received a very curt response. 'Given that we used a data-analytics-based approach, it is highly unlikely that our conclusions are incorrect.'
This is a symptom of the all too common 'digital myopia', where far too much weight is given to systems and routines being used and not nearly enough to essential aspects such as frame of reference, definition of problem, and examination of assumptions. There is a tendency towards this imbalance in much of the process around large-scale social analysis, be it polling or social media exploitation. Very often in the age of big data, scant attention is paid to meaning – especially in the early or framing stages of the analysis or the means and mechanisms by which the significance of secondary or reactive responses can be determined.
In order to achieve meaningful measurement of effect – and this is without even considering how to go about creating effects in the first place – multidisciplinary teaming is required. Yes, data analytics are essential, and any analytics team dealing with large datasets requires data scientists, machine learning specialists, and digital tools of a high standard and quality; however, none of these will be effective without the careful and decidedly analogue process of thinking through the assumptions, framing the queries, and executing the interpretation.
Assumptions and questions are crucial for setting frame of reference and designing the queries which will shape the outputs of any analysis. This requires, at the very least, specialists in formal logic, people and culture, digital culture, and – in the case of foreign audiences – linguists and ethnographers. What this means is that Army should be investing in either a native or a contracted/partnered capability in counter-intuitive disciplines like philosophy and anthropology. Rather like the military which won WWII.
And in the interpretation phase, a true multi-disciplinary approach should be used. In the same way a complex archaeological site needs people from up to a dozen different fields, the outputs of various analysis models need to be parsed and interpreted by a team consisting of experts in literature, culture and ethnography, marketing, and history, as well as technically and digitally competent individuals. In the short term, the only realistic way to do this is to create closer academia partnerships – which will need to cross multiple institutions and campuses, as each university faculty is highly distinct in approach and understanding. There is also the requirement to diversify the mental models deployed in these teams, and the only realistic way to do this is to employ more people external to the ADF who have ways of sense making and understanding which are scarce within the uniformed services.
With careful and targeted setting of questions and assumptions shaping a holistic complex data appreciation and analysis capability, we would then have a much better chance of generating the kind of learning and progress loop which will benefit and provide direction to our influence and informational efforts. Because ultimately, influence is an affective pursuit, and its only meaningful measures are likewise affective. While we have become very good at mitigating human limitations with machine processes, we should also be seeking to enhance human capabilities with machine assistance, which will mean making sure that humans guide machine assisted processes for understanding human reactions, rather than the other way around. The potential error looming on the horizon is identical to the Göbekli Tepe ‘stone age observatory’ theory, where the balance between ‘hard’ and ‘soft’ sciences is not sufficiently addressed and hard questions are glossed over or ignored in pursuit of the illusory ‘infallibility’ offered by big data.
 Martin B Sweatman and Dimitrios Tsikritis
 The Younger Dryas Event is the last glacial peak event before the end of the ice age, approximately 10,500 years ago.
 Catastrophists believe that the Younger Dryas Event was caused by an asteroid impact in North America. The prevailing theory is that melt water towards the end of the ice age changed Atlantic circulation, causing the climate to cool.
 Given this, the archaeologists at Göbekli Tepe published a detailed response in which they pointed out, among other things, that the animal motifs were most likely animal totems not star signs, that the belt buckles on the humanoid figures were just that, not comet tails, there was a strong possibility that the enclosures had been roofed, and that many of the pillars had been refurbished and moved, making use as an observatory highly unlikely. The response (J. Notroff, O. Dietrich, L. Clare, L. Dietrich, J. Schlindwein, M. Kinzel, C. Lelek-Tvetmarken, D. Sönmez: More than a vulture: A response to Sweatman and Tsikritsis. Mediterranean Archaeology and Archaeometry 17(2), 2017, 57-63) can be found here.