Harnessing generative AI for EHCPs: A deep dive into Large Language Models

We believe that generative Artificial Intelligence (AI) has transformative potential to enhance the quality and consistency of EHCPs, doing so in a way which helps Local Authorities meet children and young people's needs at a greater pace and a more personalised level.

In this article, we introduce what Large Language Models (LLMs) are, how they work, and highlight some of the key ways they can help Local Authorities.

What are Large Language Models?

"LLMs" stands for "Large Language Models" and refers to a type of AI that specialises in working with text and language. One area where this cutting-edge technology can be particularly useful is in the drafting of EHCPs, which involves analysing multiple written documents such as clinical assessments, referral forms, and panel papers. By utilising LLMs, work processes can be made more efficient, leading to better quality outcomes.

What is artificial intelligence?

AI is defined as "an entity that performs behaviors that a person might reasonably call intelligent if a human were to do something similar."

This means that if a machine or programme can carry out tasks that a human would consider intelligent, it can be classified as AI. However, some systems that seem intelligent may be following simple instructions that have been pre-programmed and are much less risky than you may think!

For example, you could describe characters in a video game as 'AI' because they can certainly seem intelligent. For example, characters in a video game may appear intelligent, but they are usually following straightforward instructions such as "if the character comes within 2 meters of the main character, then duck with a success rate of 10%".

At Invision360, we have developed a generative AI platform called VITA. VITA is powered by a sophisticated framework of rules and code to efficiently process text. This enables our platform to generate recommended content for EHCP documents.

Generative AI vs LLMs: what's the difference?

We hear the terms generative AI and LLMs used interchangeably, but what is the difference between the two? Let's break it down.

Generative AI is a broad category of artificial intelligence that refers to any AI that can create content. While some generative AI tools do not rely on LLMs, LLMs represent a subset of generative AI that specialises in generating text. Other examples of generative AI include VALL-E, which generates audio, and Stable Diffusion, which creates images.

Large Language Models

An LLM is a specific type of generative AI, specifically one which works with language. They aim to simplify human language and its understanding and generation into a set of rules and codes. The current iteration of LLMs, such as GPT-3.5, BERT, and Claude, are called "large" language models because they require a lot of computational power to run.

LLMs consist of many rules, also known as "neural networks" and "transformer architecture," to predict the next word in a sentence. They are developed by combining many different components and mechanisms that allow the model to predict increasingly refined guesses about the next word in a sentence.

Decoding LLMs: How they work

An LLM is developed by bringing together a number of different components and mechanisms that allow the model to predict increasingly refined guesses about the next word in a sentence.

1. Training data: LLMs are trained on huge amounts of data that can include books, articles, and the internet. Through this exposure to language patterns and semantics, the model learns to recognize and generate new text. During training, developers will feed a snippet of text from one of these sources and ask it to guess the next word. If it guesses wrong, the model is improved until it gets it right, most of the time.
2. Transformer architecture: Language is very complex, and LLMs use something called a transformer architecture which lets the model understand the relationship between words and sentences (e.g. ‘king’ and ‘queen’). These relationships let the LLM be better at quickly guessing the next word in a sentence.
3. Self-Attention mechanism: An important feature of transformers is what’s known as their self-attention mechanism, which allows the model to weigh the importance of different words within a sentence based on their contextual significance. This mechanism enables LLMs to grasp complex syntactic structures, many of which the human brain learns to do naturally - this also makes LLMs better at predicting the next word in a sentence, and means it is able to do so more quickly.
4. Fine-tuning: Popular LLMs (e.g. GPT-4, BERT) are trained on vast textual datasets from the internet, but they can also be fine-tuned on specific domains or to perform tasks in a specific way. At Invision360, our data scientists spend a lot of time refining VITA so that it learns specifically from best practice EHCPs (fully anonymized!) and specialist reports, so that the LLM is more focused on the areas of EHCP drafting that will lead to the highest quality.

Enhancing EHCP workflows with LLMs

LLM-based tools like Chat-GPT operate using complex sets of rules designed to generate text that could have reasonably appeared on the internet. They use a set of models and codes to make educated guesses, drawing from prior observations.

When you interact with an LLM, you'll usually receive responses that draw from its vast collection of prior interactions or closely related examples. It's important to recognise that anything created by LLMs, while usually not very creative or entirely unique, reflects a synthesis of valuable insights from diverse sources. Your first instinct when looking at an output should give you a sense of connection, recognising that the model's understanding likely stems from addressing common questions and tasks it has encountered previously. This confirms its capability to offer informed and relevant responses.

These two features make LLMs particularly useful for handling repetitive types of language tasks, making them ideal for generating a first-draft EHCP.

Generative AI has been introduced across various sectors, such as medical, financial, and social. Invision360 believes that it is now the SEND sector’s turn. In our recent blog around  the role of genrative AI in siCPs, we explain how VITA, our LLM-enabled new product can deliver higher quality EHCPs.

In conclusion

The landscape of Special Educational Needs and Disabilities is continuously evolving in response to new research, policy, and technology. We strongly believe that integrating LLMs into EHCP workflows can offer significant benefits. By working alongside SEND caseworkers and the multi-disciplinary team, the EHCP process can be improved in terms of its quality, personalisation, and efficiency. Ultimately, this can lead to better outcomes for children and young people.