9ine Insights | Latest news from 9ine

Schools are building their own applications with AI? Solving the responsibility problem

Written by Mark Orchison | Nov 13, 2026, 12:41:16 PM

Recently, I have attended a significant number of conferences in the UK, the USA and elsewhere internationally. One issue is coming up more frequently in conversations with school technology leaders: the work being created when teachers, students and support staff build their own applications using AI.

These applications are being developed through tools such as Claude and ChatGPT, or vibe coding platforms such as Lovable and Replit. Some projects have been sanctioned by the school and developed within organisation-controlled accounts and environments. Others have been created through personal accounts, without the school’s knowledge or involvement.

Often, the application starts with a reasonable idea. A teacher wants to improve a classroom activity. A department wants to simplify a process. A student builds something useful for a project. A member of support staff identifies an administrative task that could be automated.

The application works, and the person who created it wants others to benefit. This is usually where the questions become more complicated. The creator wants other teachers, students or support staff to have access. They want to add student work, personal information or confidential school data. They may want to connect the application to an existing school system so that information can be read, updated or transferred automatically.

The request then arrives in the IT department.

For the technology team, this creates a substantial piece of work that may never have been anticipated, planned or resourced. They are being asked to approve an application that someone else has built, often with limited documentation and an expectation that, because it already works, making it available should be straightforward.

Yet the school needs to apply the same discipline it would apply when considering a purchased platform. The depth of review should reflect the information involved, the access provided and the school’s intended reliance on it.

With a commercial application, there is normally a supplier to question. There may be contracts, privacy notices, information security documentation, support arrangements and evidence of testing.

With an application built in-house, much of that evidence may need to be created from scratch. The coding and hosting platforms will have their own terms, but those terms do not explain whether the application itself has appropriate permissions, handles information correctly or can be maintained.

So what is the technology team supposed to do? Who owns the application, and does the school have the rights and access needed to maintain it? Who carries responsibility if something goes wrong? Who is the “data controller” under the applicable privacy and data protection law, where that terminology applies?

What information has already been uploaded through the creator’s account? What can the development and hosting providers do with it? Can one user access another user’s records? What permissions does the application have within the school’s systems?

Then there are the engineering questions.

Who has reviewed the generated code? Who will maintain it when the creator leaves, changes role or loses interest? Who will assess requests for new features and check that an update has not introduced another problem?

If a security alert affects its PHP components or another dependency, who will establish whether the application is exposed, apply the fix and test it before the service is used again?

These are legitimate questions that need answers before the school introduces an application into its live environment. The fact that software was inexpensive or quick to create does not reduce the work needed to assess and support it.

The technical team may be the first to receive the request, but answering all these questions requires input beyond IT.

A privacy specialist needs to examine the use of personal information, the relevant legal roles and the arrangements with external providers. Cyber security expertise is needed to assess the application’s exposure and test relevant controls. Developers need to evaluate the code, its dependencies and whether it can be supported.

Academic and digital safety colleagues also have work to do. They need to consider whether the application is appropriate for its intended users, how it will be supervised, whether its outputs can be relied upon and how concerns will be raised.

Most schools do not have this full range of expertise available in-house. At 9ine, we do.

That is why I am pleased to introduce 9ine Application Assurance, for schools building and using their own applications with AI.

The service brings together our privacy, cyber security, technology, AI governance and software development expertise to help schools assess what has been built and understand what adopting it will involve.

We begin by classifying the application: its purpose, users, information, hosting, permissions, integrations and operational importance. A student prototype using synthetic information needs a different approach from an application connected to live student records or supporting a whole-school process.

The core service includes an Application Assessment, applying the scrutiny normally associated with a vendor assessment to the application the school is developing itself. For agreed individual reviews involving personal information, we also create or update the relevant Record of Processing Activities and prepare a Data Protection Impact Assessment.

Schools can add application security testing aligned to relevant OWASP Top 10 risks, followed by source code review where access to the code is available. These examine different questions: how the running application behaves, and what its code and dependencies reveal about security, quality and maintainability.

However, identifying findings is only part of the work. The school also needs people who understand what to do with them.

This is where the Diamond Formation runs through the service.

The Diamond Formation is 9ine’s governance model, bringing academic, digital safety, technology and privacy colleagues together under school leadership oversight. Each function contributes a different perspective. For operational applications, the relevant business or service owner also needs to be involved.

As part of Application Assurance, 9ine coordinates these roles. We explain the input required from each, gather their evidence and help them understand the findings and the actions that fall to them if the school proceeds.

This addresses a practical training gap. Holding a Diamond role does not automatically prepare someone to assess software developed in-house. Colleagues need support to understand how their existing obligations apply, when they need specialist advice and how their decisions affect other functions.

If use is approved, the school needs an agreed action plan with named owners, dates and evidence of completion. Leadership needs to understand the remaining risks, the resources required and the arrangements for continuing oversight.

Our role is to help schools put that process into practice, drawing on the depth of expertise needed to connect privacy, digital safety, security and engineering findings.

There is considerable value in the creativity we are seeing from teachers, students and school staff. But the request to share an application with the rest of the school needs a clear route through assessment, approval and ongoing support.

If these requests are already reaching your technology team, 9ine Application Assurance provides the specialist assessment and coordinated support to help your school decide how to proceed.

Get in touch to learn more.