Insanity Evaluation Is Characterized by a Low Inter-rater Reliability

Insanity evaluation requires to determine (1) whether the defendant is affected by a psychiatric disorder; (2) whether and to what extent that disorder also affects the defendant’s ability to understand and/or to freely will; and (3) whether there is a causal link between the mental condition and the criminal act. A recent systematic review and meta-analysis of previous studies that investigated inter-rater reliability of sanity opinions revealed agreement rates ranging from 57 to 100% and kappas ranging from 0.28 to 1.0 (24). The meta-analysis returned estimates of 0.41 (95% CI: 0.29–0.53) for sanity opinions. While this would be considered relatively strong from a mere statistical point of view, such a concordance rate is actually quite unsatisfactory when applied to the legal context, in which judges are required to reach decisions in compliance with the BARD principle.

The required evaluation of the existence of a causal link between the diagnosed mental condition and the committed criminal act may also be a factor of further discrepancy among experts. Specifically, causal link means that the crime must be an evident consequence of the underlying mental disorder: for instance, a causal link may be proven if an individual with a persecutory delusion hits a neighbor who the subject was convinced was spying on him, but not if the same individual steals a jacket from a store. The identification of this causal link is subjected to arbitrariness as, in the disregard of respect to accepted guidelines, some experts may detect a causal link, whereas others may not. Recently, the European Psychiatric Association (EPA) found that the evidence base for forensic-psychiatric practice is weak and therefore has provided recommendations for best practice (35). Moreover, the American Academy of Psychiatry and Law (AAPL) has published guidelines based on a comprehensive review of legal and psychiatric factors to be taken into account to perform insanity defense evaluations (Journal of AAPL, 2014). Finally, a recent work proposed an insanity assessment support scale based on clinical criteria (36). This instrument, however, still has to be validated. Overall, the adherence to standardized guidelines to fulfill shared and validated criteria to assess mental insanity remains to date largely unsatisfactory.

A recent retrospective study on insanity evaluations found that criminals judged as insane presented with more severe psychiatric symptoms and had a higher number of psychiatric treatments and involuntary hospitalizations as compared to those who ended up to be considered not mentally insane (37). In addition, time-bias (37) or crime-related (38) factors, more than strictly mental state related factors, appear to influence insanity opinions.

It should be emphasized that forensic psychiatric evaluation is a decision-making process in which a clinical, naturalistic element (the nosografic diagnosis) is unavoidably coupled with a juridical element (the expert’s opinion on insanity). Insanity, indeed, is not a natural data per se; as a matter of fact, the criteria for insanity stated by the law, as well as other social factors, may lead to completely different conclusions in the same case, depending on the country where the evaluation is performed (39). This makes the comparison of procedures and results across different countries very difficult, further complicating the scientific evaluation of the field.

Clinical Diagnosis Is Affected by Cognitive Fallacies/Biases

As any human being, also forensic experts may be vulnerable to cognitive fallacies. Cognitive fallacies are cognitive failures that subconsciously influence the perception and evaluation of evidence (40, 41). Cognitive fallacies are present event in disciplines in which mistakes are supposed to be essentially absent, including fingerprints and firearms analyses (41, 42) The fascinating research on cognitive biases in forensic sciences is rapidly expanding (40, 43–47), as well as their occurrence in real cases (14, 16). The first and most important result from these studies is the general lack of acceptance of the need for procedures to minimize cognitive biases and a failure to recognize susceptibility to biases (48), an effect that hereafter will be called “denial cognitive bias.” A similar bias is called “blind spot,” which refers to the tendency to recognize bias in others while remaining oblivious to one’s own biases. Critically, the denial cognitive biases and the blind spot are the lesser acknowledged biases among forensic experts (49). Another predominant cognitive bias emerging in criminal trials, as revealed by a systematic review (43) and a survey (40), is the confirmation bias, consisting in the tendency to search for, agree with or interpret information in a way that confirms one’s own presumptions (47). This cognitive bias is present in the great majority of criminal trials (16, 40, 42, 43, 50). In addition, forensic science is also affected by the adversarial allegiance effect, that is, the tendency to find pieces of evidence supporting the side that retains the expert (43, 51). Despite some legal systems, such as the Italian one, attempted to overcome this limitation by including a super-partes expert summoned by the judge (Peritus), this strategy alone is not sufficient to mitigate biases, as the Peritus also is subjected to subconscious biases, such as prejudice, denial bias, confirmatory bias and so on and so forth.

Intriguingly, cognitive biases can affect the observation of evidence, that is, the clinician’s ability to detect psychiatric symptoms as well as the conclusions, defined as the cognitive interpretations of observations, that is, the psychiatric diagnosis (14, 44–46). As an example of bias at the level of the observation, Table 1 reports the symptoms observed by three different clinicians in a recent criminal case of a woman charged for supposedly killing her own husband and with previous documentation supporting the presence of chronic alcoholism (which, in Italy, is a relevant factor for insanity evaluation): three clinicians observed three different psychiatric symptom constellations Importantly, because he did not notice any signs or symptoms of chronic alcoholism, the Peritus denied additional investigations (neuropsychological test and an MRI) to verify the potential presence of neurological damages due to chronic alcoholism and eventually he concluded for the woman to be mentally sane.

Furthermore, a representative example of cognitive bias in inter-rater reliability emerging at the level of the conclusions pertains to the famous Breivik case, in which different clinicians who observed the same symptoms reached very different diagnostic conclusions: specifically, narcissistic personality disorder combined with pseudologia fantastica as opposed to paranoid schizophrenia, obviously with major different implications for insanity (52). Another example of bias at the level of the conclusions was described in a recent paper, where self-reported information from the defendant were considered, respectively, either indicative of psychiatric symptoms or deprived of any significant meaning by different consultants (53).

Neuroscientists Believe That Neuro-Scientific Methods (in Particular Neuroimaging) Alone Are Enough to Determine Insanity

“The most pernicious error here, one that is not easy to spot, is the claim that because the amygdala is the fear center, activity there indicates that the defendant was experiencing high levels of fear” (66). Authors who express this kind of criticisms assume that neuroscientists who promote the application of neuroscience methods in the forensic context, believe that neuro-scientific methods, and in particular neuroimaging, can be used in isolation (i.e., independently from the presence of any clinical symptoms) to claim insanity. In other words, the worry is that neuroscientists intend to replace the clinical assessment of insanity with neuroscience and neuroimaging methods. This criticism is based on a misinterpretation of the neuroscience position, a logical bias called “strawman fallacy.” Indeed, it provides a distorted version of the neuroscientist opinion, as neuroscientists are not claiming that brain imaging (or any other laboratory test) alone is sufficient nor that it should replace clinical interviews; rather, neuroscientists sustain that criteria for responsibility are and should remain based on clinical factors (22) and that brain scan exams may provide an objective support to the clinical diagnosis (15). Of note, a brain abnormality per se is not necessarily sufficient for determining insanity (15). For instance, the Italian serial killer Gianfranco Stevanin, despite a macroscopic traumatic lesion in the frontal lobe, was considered criminally liable because his ability to do otherwise was spared (15). Claiming that applying neuroimaging to insanity evaluation lacks of incremental validity above and beyond extant information (18, 21, 67) is simply untrue, as neurobiological evidence does contribute to solve diagnostic disputes and to obtain a more reliable diagnostic profile (68). As recently demonstrated, neuroimaging has indeed the potential to delineate also distinct and highly reproducible neuroanatomical subtypes within the same pathology (69). Thus, the incremental value of neuroscience lays in its usefulness to reduce the degree of intrinsic variability in insanity assessments (70). Of course, the relevance of neuro-scientific results in evaluating insanity may vary from case to case depending on individual features (71). Finally, any determinism should be strictly avoided (15).

Neuroscientists Want to Infer Insanity From Brain Structure/Activation

“A structural MRI reveals a brain defect in the frontal lobe, which is then used to justify the assertion that because of the defect, the person has impaired impulse control or impaired rationality” (66). Similarly, “Neuroimaging experts often posit direct connections between brain data and criminal actions, arguing that such connections form the foundation for an insanity defense” (21). The above statements are two representative examples from recent literature of a major misconception of the utilization of neuro-scientific methods. As a matter of fact, neuroscientists are aware that brain imaging data cannot prove any direct causal connection between brain structure/function and criminal behavior. Indeed, we recently remarked that “anatomo-clinical correlations can be assessed between brain region and cognitive functions but cannot be assessed between a brain region and criminal behavior, as there is no specific brain region involved in complex behaviors such as criminal violence” (15). In other words, the anatomo-functional correlation should be assessed between neuroimaging data and neuropsychological test results for cognitive or behavioral functions that are relevant to comprehend the criminal act, such as impulse (dis)-control or (deficits in) moral reasoning. Furthermore, the causal link between behavioral alterations and the committed crime is another fundamental step in the evaluation of insanity (36). Thus, this criticism is affected by the “strawman” cognitive fallacy and violates each one of the four rules stated in recent peer-reviewed indications regarding the utilization of neuroimaging in court (15).

Neuroscientific Techniques Do Not Comply With the Daubert Criteria

“Many scientists have debated the relevance and admissibility of neuroimaging data to the insanity defense” (21). In our view, this kind of criticism is too general and vague. For instance, it is not clear if it refers to structural or functional neuroimaging data. In addition, there are techniques that clearly comply with the Daubert criteria: one example is the MRI structural technique called Voxel Based Morphometry (VBM), which is used to analyze the structure of the brain in order to detect subtle brain changes that are not visible at gross inspection (72–74). These techniques, whose error rate is known both at the level of group and single individual analyses (75–77) and are fully accepted by the scientific community, unfortunately are sharply criticized when applied to the forensic field. Importantly, these criticisms are not directed to the techniques per se, but to the ways these techniques are believed to be utilized and their results to be interpreted. For instance, critical authors believe that VBM would be applied to find the neural basis of insanity and that any detected abnormality would be used to sustain insanity, regardless of the presence of clinical symptoms. On the contrary, neuroscientists clearly stated that VBM can only detect subtle brain abnormalities and that every inferences made on insanity is the result of a deductive reasoning based on stringent anatomo-clinical correlations (between the symptoms presented by the defendant and the anatomical localization of the abnormal brain region) and on the convergence of evidence, that of course are independent steps from VBM itself (15).

The Study of the Brain Is Useless for Insanity Purposes as the Brain Is Dynamic in Nature

“There is the problem of time: because people do not walk around wearing scanners, neuroimaging evidence presents information regarding brain structure or function after the fact” (66). The above statement synthetizes the concern that brain imaging examinations are performed sometimes months, if not years, after the criminal facts have occurred. Although in most cases insanity assessments do take place a long time after the criminal acts, there are several aspects that need to be considered. In the first place, the time lag between the facts and the examination of the defendant applies to any component of the insanity assessment, including the psychiatric examination. As a matter of fact, psychiatric symptoms may rapidly change over time, both as a spontaneous evolution of the disorder and/or because of the treatments that are administered to the subject (78–80). Thus, establishing months later the “state of mind” of an individual at the time of the facts is typically an exercise based on assumptions and the evaluation of indirect pieces of evidence. Second, while it is undeniable that the brain is plastic, clinically relevant brain structural changes are obtained with exercise or rehabilitation, aging, drug or alcohol abuse or pathological processes (81–84). It is highly unlikely, if not impossible, that an individual’s brain may undergo clinically significant modifications over a few months just because of the elapsed time, in the absence of any intervening brain accident or insult (e.g., infarct, head trauma, tumor). To clarify this issue with an example: let us assume that a MRI scan exam, for instance, shows a significantly smaller gray matter volume in the orbito-frontal regions, known to play a crucial role in impulse and behavioral control, in an individual who committed an impetus crime as compared to gender- and age-matched controls. That difference is real and it is irrelevant that the MRI scan had been performed a year after the facts, unless, of course, the subject in the meantime had suffered an accident that could have resulted in a loss of volume (e.g., severe head trauma, vascular events, resection of a tumor). Obviously, such a major event would be documented, especially given that the defendant would have been in custody since the very first days after the facts. An identical reasoning applies to functional brain data. Though certainly more susceptible to variability over time than structural data, measures of brain functional activation in response to specific tasks remain relatively stable within the same individual over time, unless there have been other interfering factors, such as assumption of psychotropic medications or illicit drugs, or pathologic events as mentioned above (85, 86). Thus, a brain anatomical alteration or a dysfunctional response in regions or circuits that modulate behavioral responses or cognitive functions relevant for insanity assessments, may be a valid element even if collected at distance of time after the facts. However, as stressful life events may affect brain morphology (87, 88), prolonged incarcerations theoretically could be associated with brain structural and functional changes. To what extent such changes may affect cortical regions that are crucial for self-determination and are therefore relevant in insanity assessments remains to be established.

On the other hand, the elapsed time may play a different role in the case that the brain lesion present at the time of the criminal act had been removed or repaired when the individual undergoes the insanity assessment. For instance, in a published case of acquired pedophilia, pedophilia emerged as a symptom of a clivus chordoma, a tumor of the notochord (89). While the defense consultants had the opportunity to examine the defendant prior to the surgical removal of the tumor, the Periti did not. Thus, they could not appreciate the variety of symptoms and signs that the consultants had detected at the time of the first examination. Eventually, the Periti concluded that the defendant was not insane at the time of the crime .

The Study of the Brain Is Problematic in Insanity Evaluation as Brain Images Should Be Interpreted

“A functional or anatomical anomaly is interpreted as being a handicap only insofar as the behavior it may produce is considered such” (18). We do agree that while results from neuro-scientific techniques (neuroimaging in particular) may inform on the biological basis for insanity, they cannot be themselves considered a proof of disability. This is necessary to avoid determinism. Neuro-scientific results should be interpreted taking into consideration the behavioral constellation of symptoms manifested by the defendant, with the advantage that neuro-scientific results cannot be malingered, unlike behaviors. For instance, a lesion in the frontal lobe can be the cause of a dis-inhibited and depraved sexual behavior conferring the base for insanity [see case 1 described in (15)], while a very similar lesion in the frontal lobe can be present in a defendant charged for murder during extreme sexual acts and feigning frontal-lobe induced dis-inhibition, thus leading to the decision of absence of insanity [see case 2 described in (15)]. Neuro-scientific results, including imaging data, may provide an objective contribution to solve controversies on insanity among experts, as described in a recent published paper (14). Unfortunately, clinical signs and symptoms may be interpreted in different ways leading to cognitive biases at the level of the conclusion of the Human Expert Performance model (45, 46) described in the first section of this paper. Neuroscientists who support the usefulness of neuroimaging are well-aware of its drawbacks and limitations: for this reason, we recently proposed objective indications on how to interpret neuroimaging results (15).

The Inference Group to Individual Cannot Be Done

“Science generalized, population level knowledge of a phenomenon does not necessarily provide an appropriate empirical foundation for making inferences about the instantiation of that phenomenon in any given individual” (67); “Virtually all brain scans rely on group data to determine the presence or absence of irregularities. The law, in contrast, typically relies on an idiographic approach, whereby each case is decided on its unique merits and facts” (21). The first weakness of this criticism is that it does not distinguish between macroscopic and subtle brain alterations. Indeed, when gross brain alterations, such as brain tumors, traumatic injuries, advanced dementias, are considered, this statement is completely unfounded. In all these cases, which are the great majority, clinicians routinely make inferences at the level of the single individual in everyday clinical practice (i.e., when deciding whether to surgically remove the tumor, whether or not include a patient in a clinical trial, etc). Nevertheless, the use of neuroimaging has been criticized also in these cases (90). A distinct consideration is required in the case of subtle brain alterations, such as regional differences in neuronal density or fiber tracts, that can be detected only by performing a statistical analysis between group of patients and group of controls (91–94). Unlike neurological disorders, which usually are accompanied by brain alterations that can be easily revealed by routine laboratory exams, including tumors, strokes, atrophy or inflammatory processes, psychiatric disorders typically do not show gross morphological abnormalities. However, specific psychopathological features may indeed be associated with more subtle alterations in selected cortical or subcortical structures, as well as in neural circuits, that are devoted to emotional processing, modulation of aggressive behavior or moral judgment (11, 95–99). As they regard psycho-(patho)logical features, these abnormalities may very well be trans-diagnostic (100–102) and consistent across different ethnic groups (103). Though most of the above observations have been provided by group comparisons, in recent years researchers have aimed at enabling inferences to be made at the level of the individual subject (104–108). Further, the error rate of these single case analyses also were calculated in order to fulfill the Daubert criteria (75, 76). In addition, guidelines on how to correctly interpret these data were developed to prevent erroneous conclusions (15). Noteworthy, the scientific community is working to improve translation of neuroimaging findings to the clinical settings through the implementation of algorithms able to identify brain abnormalities in individual patients. Indeed, eight tools already have been released freely online to support clinical diagnoses : clinicians can upload the MRI scans of an individual patient; an automated and validated algorithm will analyze the brain to detect potential structural alterations and a detailed report will be generated and sent to the clinician. Finally, recent advances have proposed a normative modeling in neuroimaging analyses of psychiatric disorders (110, 111). According to this approach, the degree to which each individual brain deviates from the normative pattern is calculated so that these deviations can be mapped in each individual and psychopathology can be conceptualized as deviation from normative patterns (110–113). In other words, the normative modeling provides a way to quantify and characterize the manner in which the brain of different individuals deviates from the expected normal brain (110, 113). Authors clearly declared that this approach cannot indicate directly whether the obtained deviations are clinically relevant (110), and for this reason neuroimaging results should assume clinical significance only when associated with clinical symptoms. In summary, group to individual inferences are possible even when structural abnormalities are not grossly evident and are revealed by rigorous and sophisticated statistical analyses.

Neuroimaging Are Useless as There Is no Known Neural Basis for Insanity

“Large-scale normative imaging data on individuals found insane have yet to be collected (…) this means that data are not sufficiently advanced to bear direct relevance to insanity determination” (21). Insanity is not a specific syndrome or disease that can be investigated as a unified entity, but rather may derive from a variety of diseases with a different etiology. For instance, insanity may be due to dementias, brain tumors, psychoses, traumatic brain injuries. Of course, the brain regions involved in these pathologies may widely differ, thus it is highly unlikely to find a common neural substrate invariably shared by individuals found to be insane. A further complication is that insanity is a condition defined by the law, not a medical diagnosis. Specifically, it is up to the judge to ultimately decide whether or not the defendant is insane: such a decision not necessarily reflects the expert conclusions nor, as discussed above, the experts from the different sides may necessarily agree on the presence of insanity in the first place. Thus, a search for the neural basis of insanity as such would be affected by major conceptual caveats, as the very same individual could be considered completely insane by an expert and fully capable by another one. Rather, a promising strategy in this direction is represented by the search for the neural basis that may underlie abnormal/antisocial behaviors in specific categories of offenders including rapists (97), batterers (96, 114), or murderers (11, 98). Unfortunately, biased by the logical fallacy called circular reasoning, the above studies are criticized as well, as they are believed to look for a neural basis of insanity (21).

What Neuroscientists Do Not Claim