AI is all the rage nowadays. Rage about its use, and rage to use it. Artificial intelligence is the wave of the future, some proclaim. Others call it a dud. The fact is, it’s not quite either. It’s a potentially powerful tool on which we’re placing ridiculously high expectations, expectations it’s not designed to fulfil. We want it to be a super-genius, analyzing data and outputting new knowledge at a rate and with an insight people can’t match. In actual fact, the mechanism of AI just isn’t able to do that. It’s very good at certain tasks and logically incapable of others.

What AI is Good At

‘Artificial Intelligence’ as we’re using it nowadays refers to ‘neural net’ programs. These programs work by taking an input, running it through many steps of calculations as prescribed by its programming, and then outputting the answer, just like any other program (often with a little nudging from a pre-set list of numbers to add apparent randomness). What’s different is how they are created. Rather than being written line-by-line by a human, these programs are built by making a program that adjusts itself in response to input from the user. Theoretically all these adjustments are deterministic and thus foreseeable by the programmer, but in actual fact the adjustments and their effects quickly becomes impossible for a human to keep track of (due to sheer volume), resulting in a program that is effectively black boxed- but also potential very, very good at matching the input-ouput schema it was trained to match.

Artificial intelligence has several strengths. First, it’s very good at generating something that looks like the data set it was trained on. Generative AIs are an example of this capacity. I feed in text, and the AI spits out text which could plausibly answer or continue that text I fed in. The crucial point to recognize here is that the AI isn’t pursuing or even capable of considering informational accuracy; it doesn’t analyze text on the level of information. Instead, it analyzes the text as a series of letters (actually, their code-equivalents), without reference to meaning, and outputs a sequence of letters which iterates upon the pattern I’ve started. The seeming generation of meaning may be convincing, but it’s merely a by-product of the meaningfulness of the text it was trained on, whose pattern it is imitating. And yes, it is just imitating a pattern- a pattern is much, much larger than the single case, preventing the pattern from reiterating and thus becoming apparent.

(The pattern, incidentally, is the sum of the calculations performed by the matured neural net to transform input into output.)

AI has great potential also to point out patterns and statistical points of interest in data. It has particular potential in areas of low individual complexity but high need for repetition- relatively simple analysis that must be repeated too often to be practical for a human- or for discovering patterns too complex to be found by simply brute-forcing possibilities, as with protein folding, where AI can be molded towards tendencies human analysis misses due to AI not needing to find an actual tendency or the causes of it, given it just imitates the results of the tendency without ever conceptualizing anything.

Because it does not analyze the meaning of the data, it cannot be trusted to choose worthwhile patterns or points of interest, but it can point them out, allowing humans to go back over the data, consider its meaning, and decide whether the pattern noticed is useful, meaningful, or otherwise. So an AI might be useful in noting a statistical correlation between certain image elements and a certain disease, but the researcher will need to check if the AI is finding a significant correlation, such that the image element can be used to find the disease in future, or something meaningless: a correlation between the use of a particular machine and having the disease, a technical error in how the some batches of the images were saved to digital format, etc.

A Big Problem

The AI can only note correlations in the data it has; it is incapable of getting more data. An AI can be programmed to take in X categories of data and Y amounts; it can be programmed to request such-and-such of both for use in its calculation, perhaps triggered by certain elements in the input. That programming, however, is absolute.

Let’s compare you or I with the AI. Imagine a human and an AI sitting down to the same problem: get the box open. The AI was trained in an environment with scissors as its means of opening boxes; you have your life experience to draw on. The room you’re in doesn’t have scissors.

The AI will run down through its calculations. Possibly it has some method of opening boxes that rises from ‘no-scissors’ as an input. It tries this; it fails. Meanwhile, you, noting the lack of scissors and the fact that the box is in fact a wooden crate nailed shut, go get a clawhammer, pry it open, and walk away with the prize. The AI, whose input never included the clawhammer, keeps trying to open the crate, indefatigable, and never succeeds.

Or return to the problem mentioned earlier, with using medical scans to diagnose. The AI found a correlation between a certain feature of the scan and the diagnosis. Unless the programmer trained it to check for the possibility, however, the AI will never check its result to be sure the feature it has identified isn’t the stamp the researcher put on the image to verify it belongs to a sick person or a technical error endemic in machines of a certain region which at the time of imaging had an epidemic of the disease in question. Both would explain the correlation, but the AI can’t consider either.

AI training accounts for this fact. A neural net is trained via making small, pseudo-random changes (random from the human perspective; deterministic from an omniscient perspective) in the code, testing if those changes improve the code, and keeping the ones that do improve it. Repeated enough times (millions, billions), and you get the immensely complex neural nets of generative AI. This training process (properly done) does what it is supposed to do: it shapes the AI to produce the desired output, nothing more, nothing less. (The bizarre results of training going haywire stem in large part from the sheer complexity of the system (meaning butterfly effect is quite possible) and partly from the incomprehensible complexity of the calculation, making all disturbances impossible to trace to their source.)

AI is thus very, very good at imitating a pattern. However, in order to imitate that pattern, it needs to be trained to the pattern. The more general the pattern, however, the more data is required to train towards it. A specific pattern, such as ‘1001110’, needs only that single example. If I trained an AI to spit out that pattern, I would only need that pattern. However, if I want to train the AI to imitate a more general pattern (plausibly hallucinate, because it does not even perceive meaning), I will need to provide more and more specific examples of that pattern. The AI needs more training data because its calculation needs to be weighted to ignore irrelevant factors and analyze (in the case of generative AI, imitate) relevant ones.

The amount of training data a generative AI needs is proportionate to the complexity of the pattern it seeks to imitate. Every example in the data will be specific, and so to avoid regarding the specific aberrations of each example as parts of the general pattern, the AI will need to have mass counter-examples. If I train a generative AI exclusively on sentences starting with ‘the,’ the AI will naturally generate sentences starting with ‘the.’

Language is tremendously complex. Generative AIs are very good at certain types of language, particularly when meaning is irrelevant or small alterations don’t massively change meaning (hence why they’re very bad for legal applications, as they tend to hallucinate sources). They’re very good at the general pattern of language. However, they only got to be so good at imitating because of the mammoth amounts of data poured into them.

AI learns by brute force. It takes all the data, mashes its calculation against the mass, and comes out with something that produces whatever patterns it saw in the data, right or wrong. If the data omitted any factor, that factor never comes into consideration, however relevant. If you set an AI to determine optimum body-weight relative to age but omitted all height data, it would give you a result. The result would just ignore height data completely, nulling the usefulness of the result (unless you controlled for height in the input data, making sure they were all of the same height).

AI cannot generalize, not really. It will happily apply its pattern to any input you give it, but it cannot and will not consider whether its pattern needs modification, if it’s applicable to the current input.

Human Cross-Referencing

Give a human a small amount of data, and he can generate a theory. This theory will cross-reference his experience and knowledge, extrapolating and applying various patterns; hopefully, it will be presented as a theory, as a possibility yet untested. Train an AI on that same data, and it will give whatever result you told it was the right output; give a pre-trained AI that same data, and it will apply its singular pattern to that data, regardless of whether the pattern works. People can develop new patterns and apply old ones; AIs can either develop a new pattern (whose output is determined by the desired result, not the input) or apply an old one.

A human can look at incomplete, spotty data from a very complex topic and develop a theory. An AI will look at the same data and present its analysis, without a moment of hesitation or interpretation, but it will completely ignore the complexity of the topic. It will assume that the data it has is the complete picture; it cannot do anything else, by virtue of the logic of its construction. Moreover, if you’re training it, it will stop at the first pattern which yields the response you tell it to yield. If the amount of data it was trained on is disproportionate to the complexity of the topic (read: not massively larger than a human needs to find patterns), that first successful pattern may very well be wrong, an answer which works with the training data but will not consistently work elsewhere. This result is, of course, verifiable only by checking it on other cases (i.e. training it with more data), since we can’t read what the AI is actually doing, due to its complexity and the obscurity of its logic.

Especially in Medicine

As some may know, my father is a doctor, and I work for him part-time, editing his blogs and writing a weekly series for his website. I therefore have some vested interest in understanding how AI applies to medicine. Today’s article, indeed, is eminently applicable to AI in medicine.

Medicine is an immensely complex field. Every person has potentially billions of parameters a medical examination could look at, millions of which are plausibly relevant, thousands of which are plausibly observable, hundreds of which a singular doctor might be able to consider. Every person has an immensely complex dietary history, a person microbiome, a personal genetic code, a personal history of toxin exposure, a personal stress history, a personal spiritual health, a personal willingness to do each treatment, etc.

A full-doctor medical AI would need to account for many, many factors. Probably, to be practical, it would need also to account for lacking much information in each patient, in inconsistent areas. It would even need to account for omissions, intentional or unintentional, in the information provided by the patient. So each patient would be immensely complex.

Remember, however, that the more complex a topic, the more data an AI needs to be properly trained. If every person is so complex, the pattern governing treatment of all these people must be even more complex. The amount of data needed, therefore, must be vast beyond measure. Possibly all the medical records in the US might be a good start (imagine that breach of privacy!), but those records are flawed, missing much of the relevant information. The AI’s pattern, if based on that documentation, would lack connection to much of what goes into keeping somebody healthy or healing their sickness.

Oh, and another problem….

Conclusion

AI is powerful, but it has limits built into the architecture of its design. An AI can produce a viable pattern only if it has data proportional in breadth and complexity to the complexity of the topic it is dealing with. It must have much, much more data than a human would, to get similar insight, simply because it does not analyze meaning. It cannot relate meaning to meaning; it must connect the ephemera of the meaning, the symbols, to other ephemera, rather than getting to the essence. AI is therefore incapable of truly matching humans in areas whose complexity vastly exceeds the data available.

God bless.