Brain-reading

Identification of complex natural images is possible using voxels from early and anterior visual cortex areas forward of them (visual areas V3A, V3B, V4, and the lateral occipital) together with Bayesian inference. This brain reading approach uses three components:[3] a structural encoding model that characterizes responses in early visual areas; a semantic encoding model that characterizes responses in anterior visual areas; and a Bayesian prior that describes the distribution of structural and semantic scene statistics.[3]

Experimentally the procedure is for subjects to view 1750 black and white natural images that are correlated with voxel activation in their brains. Then subjects viewed another 120 novel target images, and information from the earlier scans is used reconstruct them. Natural images used include pictures of a seaside cafe and harbor, performers on a stage, and dense foliage.[3]

In 2008 IBM applied for a patent on how to extract mental images of human faces from the human brain. It uses a feedback loop based on brain measurements of the fusiform gyrus area in the brain which activates proportionate with degree of facial recognition.[4]

In 2011, a team led by Shinji Nishimoto used only brain recordings to partially reconstruct what volunteers were seeing. The researchers applied a new model, about how moving object information is processed in human brains, while volunteers watched clips from several videos. An algorithm searched through thousands of hours of external YouTube video footage (none of the videos were the same as the ones the volunteers watched) to select the clips that were most similar.[5][6] The authors have uploaded demos comparing the watched and the computer-estimated videos.[7][8]

In 2017 a face perception study in monkeys reported the reconstruction of human faces by analyzing electrical activity from 205 neurons.[9][10]

In 2023 image reconstruction was reported utilizing Stable Diffusion on human brain activity obtained via fMRI.[11][12]

Brain-reading has been suggested as an alternative to polygraph machines as a form of lie detection.[13] Another alternative to polygraph machines is blood oxygenated level dependent functional MRI technology (BOLD fMRI). This technique involves the interpretation of the local change in the concentration of oxygenated hemoglobin in the brain, although the relationship between this blood flow and neural activity is not yet completely understood.[13] Another technique to find concealed information is brain fingerprinting, which uses EEG to ascertain if a person has a specific memory or information by identifying P300 event related potentials.[14]

A number of concerns have been raised about the accuracy and ethical implications of brain-reading for this purpose. Laboratory studies have found rates of accuracy of up to 85%; however, there are concerns about what this means for false positive results among non-criminal populations: "If the prevalence of "prevaricators" in the group being examined is low, the test will yield far more false-positive than true-positive results; about one person in five will be incorrectly identified by the test."[13] Ethical problems involved in the use of brain-reading as lie detection include misapplications due to adoption of the technology before its reliability and validity can be properly assessed and due to misunderstanding of the technology, and privacy concerns due to unprecedented access to individual's private thoughts.[13] However, it has been noted that the use of polygraph lie detection carries similar concerns about the reliability of the results[13] and violation of privacy.[15]

The Emotiv Epoc is one way that users can give commands to devices using only thoughts.

Brain-reading has also been proposed as a method of improving human–machine interfaces, by the use of EEG to detect relevant brain states of a human.[16] In recent years, there has been a rapid increase in patents for technology involved in reading brainwaves, rising from fewer than 400 from 2009–2012 to 1600 in 2014.[17] These include proposed ways to control video games via brain waves and "neuro-marketing" to determine someone's thoughts about a new product or advertisement.

Emotiv Systems, an Australian electronics company, has demonstrated a headset that can be trained to recognize a user's thought patterns for different commands. Tan Le demonstrated the headset's ability to manipulate virtual objects on screen, and discussed various future applications for such brain-computer interface devices, from powering wheel chairs to replacing the mouse and keyboard.[18]

It is possible to track which of two forms of rivalrous binocular illusions a person was subjectively experiencing from fMRI signals.[19]

When humans think of an object, such as a screwdriver, many different areas of the brain activate. Marcel Just and his colleague, Tom Mitchell, have used fMRI brain scans to teach a computer to identify the various parts of the brain associated with specific thoughts.[20] This technology also yielded a discovery: similar thoughts in different human brains are surprisingly similar neurologically. To illustrate this, Just and Mitchell used their computer to predict, based on nothing but fMRI data, which of several images a volunteer was thinking about. The computer was 100% accurate, but so far the machine is only distinguishing between 10 images.[20]

The category of event which a person freely recalls can be identified from fMRI before they say what they remembered.[21]

December 16, 2015, a study conducted by Toshimasa Yamazaki at Kyushu Institute of Technology found that during a rock-paper-scissors game a computer was able to determine the choice made by the subjects before they moved their hand. An EEG was used to measure activity in the Broca's area to see the words two seconds before the words were uttered.[22][23][24]

In 2023, the University of Texas in Austin trained a non-invasive brain decoder to translate volunteers' brainwaves into the GPT-1 language model. After lengthy training on each individual volunteer, the decoder usually failed to reconstruct the exact words, but could nevertheless reconstruct meanings close enough that the decoder could, most of the time, identify what timestamp of a given book the subject was listening to.[25][26]

Statistical analysis of EEG brainwaves has been claimed to allow the recognition of phonemes,[27] and (in 1999) at a 60% to 75% level color and visual shape words.[28]

On 31 January 2012 Brian Pasley and colleagues of University of California Berkeley published their paper in PLoS Biology wherein subjects' internal neural processing of auditory information was decoded and reconstructed as sound on computer by gathering and analyzing electrical signals directly from subjects' brains.[29] The research team conducted their studies on the superior temporal gyrus, a region of the brain that is involved in higher order neural processing to make semantic sense from auditory information.[30] The research team used a computer model to analyze various parts of the brain that might be involved in neural firing while processing auditory signals. Using the computational model, scientists were able to identify the brain activity involved in processing auditory information when subjects were presented with recording of individual words.[31] Later, the computer model of auditory information processing was used to reconstruct some of the words back into sound based on the neural processing of the subjects. However the reconstructed sounds were not of good quality and could be recognized only when the audio wave patterns of the reconstructed sound were visually matched with the audio wave patterns of the original sound that was presented to the subjects.[31] However this research marks a direction towards more precise identification of neural activity in cognition.

Some researchers in 2008 were able to predict, with 60% accuracy, whether a subject was going to push a button with their left or right hand. This is notable, not just because the accuracy is better than chance, but also because the scientists were able to make these predictions up to 10 seconds before the subject acted – well before the subject felt they had decided.[32] This data is even more striking in light of other research suggesting that the decision to move, and possibly the ability to cancel that movement at the last second,[33] may be the results of unconscious processing.[34]

John Dylan-Haynes has also demonstrated that fMRI can be used to identify whether a volunteer is about to add or subtract two numbers in their head.[20]

Neural decoding techniques have been used to test theories about the predictive brain, and to investigate how top-down predictions affect brain areas such as the visual cortex. Studies using fMRI decoding techniques have found that predictable sensory events[35] and the expected consequences of our actions[36] are better decoded in visual brain areas, suggesting that prediction 'sharpens' representations in line with expectations.

It has also been shown that brain-reading can be achieved in a complex virtual environment.[37]

Just and Mitchell also claim they are beginning to be able to identify kindness, hypocrisy, and love in the brain.[20]

In 2013 a project led by University of California Berkeley professor John Chuang published findings on the feasibility of brainwave-based computer authentication as a substitute for passwords. Improvements in the use of biometrics for computer authentication has continually improved since the 1980s, but this research team was looking for a method faster and less intrusive than today's retina scans, fingerprinting, and voice recognition. The technology chosen to improve security measures is an electroencephalogram (EEG), or brainwave measurer, to improve passwords into "pass thoughts." Using this method Chuang and his team were able to customize tasks and their authentication thresholds to the point where they were able to reduce error rates under 1%, significantly better than other recent methods. In order to better attract users to this new form of security the team is still researching mental tasks that are enjoyable for the user to perform while having their brainwaves identified. In the future this method could be as cheap, accessible, and straightforward as thought itself.[38]

John-Dylan Haynes states that fMRI can also be used to identify recognition in the brain. He provides the example of a criminal being interrogated about whether he recognizes the scene of the crime or murder weapons.[20]

In classification, a pattern of activity across multiple voxels is used to determine the particular class from which the stimulus was drawn.[39] Many studies have classified visual stimuli, but this approach has also been used to classify cognitive states.

In reconstruction brain reading the aim is to create a literal picture of the image that was presented. Early studies used voxels from early visual cortex areas (V1, V2, and V3) to reconstruct geometric stimuli made up of flickering checkerboard patterns.[40][41]

EEG has also been used to identify recognition of specific information or memories by the P300 event related potential, which has been dubbed 'brain fingerprinting'.[42]