Editor’s note:  On Tuesday March 18, 2025, the New Hanover County Board of Education will meet to discuss the inclusion of artificial intelligence in security systems across the county.  We invited two school board members – Pat Bradford and David Perry  - to give us their views on this discussion.  Ms. Bradford’s perspective is here.  Mr. Perry's thoughts are below.

Tomorrow (March 18, 2025) the New Hanover County Board of Education will be conducting a Town Hall concerning our school district’s participation in the AI School Safety Pilot Program.  The Town Hall will begin at 6 pm in the Board of Education Center at 1809 S 13th St, Wilmington, NC 28401. The community will be granted the opportunity to ask questions from the AI vendor and to just speak their minds.  I hope you will make time to attend.

The 2002 film, Minority Report, stars Tom Cruise and is a loose adaptation of Philip K. Dick’s 1956 novella, The Minority Report. There are some plot differences, and so I am going to stick to talking about the movie.  In 2054, the city of Washington, DC has been conducting a 6-year pilot crime prevention effort called “Precrime.” By finding a group of 3 mutant humans that have the gift of precognition, the Precrime unit can infallibly predict murders before they happen and arrest the would-be perpetrators before the crime is committed. However, the fact is that the predictability of these would-be murders is not actually as infallible as the owners of Precrime let on.  Occasionally one of the “precogs” disagrees with the other two and issues a “minority report.” Because the Precrime pilot is about ready to go national, this fact is hidden from the public. By the end of the movie, Precrime is exposed, and we discover that human free will is simply non-predictable. The reason I bring up the movie is because its themes are eerily similar to what is being proposed with the AI School Safety Pilot Program. 

Similar Motivations:

We all want to keep ourselves and our families safe, particularly our children.  Even though President Reagan famously warned that “the nine most terrifying words in the English language are ‘I’m from the government and I’m here to help,” most people are not self-reliant and do rely on our government to deter crime and to do their best to secure our unalienable rights to life, liberty, and the pursuit of happiness. It is natural for us to hope that our government could accurately predict crime and stop harm before it happens. But is that a realistic hope, and, if so, what price do we have to pay for that hope?

In the Minority Report, mutant humans with precognition provide the hope for nearly infallible prediction of human behavior.  However, while mutant “precogs” are pure science fiction, AI provides the hope, that through surveillance, data collection, and continual machine-learning, it can provide that level of nearly infallible prediction that will greatly help in keeping our children safe.

While all this seems increasingly possible, we still have to ask what the cost is for this level of security and protection?  Our Founding Fathers recognized that while a “police state” would certainly provide more security, the essential liberty and privacy rights of the people would have to be sacrificed. To protect against this type of tyranny, they wrote prohibitions against this sort of government conduct in our Bill of Rights.   The 4^th Amendment guarantees our right to be free from warrantless searches of our person and our property. People are considered innocent until proven guilty, and warrants are not issued unless probable cause has been established. 

AI turns the constitutional model on its head.  Everyone is a potential subject. Everyone is actively watched and tracked. Everyone is considered as potentially guilty until proven otherwise.

The Specifics:

Before we go further, I need to outline the specifics of the pilot program. The AI School Safety Pilot Program was initially formulated by state senators Michael Lee (New Hanover County) and Steve Jarvis (Davidson County) a couple of years ago.  It allocates $3.2 million dollars to New Hanover County Schools and $2 million dollars to Davidson County Schools to implement and test AI driven security measures in our school districts in a 2-year pilot.  The initial state grant legislation granted a lot of latitude to our school districts when it comes to the choice of vendor and systems capability.  However, late in 2024, the NC General Assembly amended the grant legislation as an add-in stipulation to SB382 – The Hurricane Hele Relief Bill (See page 107).  Now our school district must pick a vendor and an AI system that is capable of the following functionality:

1. Threatening Object Detection.
2. Intruder Detection.
3. Person Down Detection.
4. Door Open Detection.
5. Tag and Track.
6. Facial Recognition.
7. Forensic Face Search.
8. License Plate Reader.

The amended legislation also requires that both school districts must use the same AI vendor.  Since Davidson County recently selected Eviden (a French Company) as their vendor, we are required to use Eviden as well. However, it should be noted that Davidson County never really had a choice either. No other AI vendor offers a system that meets the 8 specified functional capabilities. Both of our school districts had our hands tied.

If we decide to participate in the pilot, Eviden will start installing, configuring, and testing its system in one elementary, one middle, and one high school in our district.  Servers will be placed in each school and will be connected to all of the existing cameras we have in hallways, outdoor spaces, and common areas.  There are no cameras in bathrooms or classrooms, and none will be added to these areas.  Once the job is done at these three schools, Eviden will proceed to do the same at our other 43 schools and 4 district office locations.

Functional Capabilities:

Perhaps to avert the public’s fear about their children being tracked constantly by the AI System, Eviden has proposed that they be will not implement the Facial Recognition capability of their system. While the system has the capability of creating a photographic, biometric, and historical record for every student, staff, and citizen that enters school property, we are being assured that this feature will remain off. I asked how Intruder Detection would work if the system didn’t know who is supposed to be in certain parts of the school if the system didn’t know who the staff and students that are supposed to there are, and what they look like?  I didn’t get a satisfactory answer.

Threatening object detection is supposed to identify guns, knives, and other weapons. However, it should be noted that the system can only identify brandished weapons, and not weapons concealed in a book bag or underneath clothing.  One wonders how much protection this limited functionality provides.

Finally, their VYZE AI System was originally designed as a cloud product.  Due to recent data breaches and the growing ability of quantum computers being able to hack into cloud systems, the decision was made to install local AI servers at each school.  Presumably these systems would be disconnected from public internet access.  However, it is natural to ask how this AI system is supposed to learn if it’s disconnected from all the AI systems in our school district?  How is it supposed to get updates?   Eviden has provided an option to create a district-wide confederation server that would presumably reside at a central NHCS location and link to all the individual school servers.  However, that would cost the district additional money, not only to Eviden, but also for the point-to-point communication wiring we would need for secure communications between the confederation server and all the satellite servers.

Bait & Switch?

While I am not an AI Expert, I have been a Software Engineer for the last 30 years.  I am trained to distinguish the “hype” from the actual functionality of software products.  I am not as concerned about what the Eviden Vyze system will do out of the box, based on the promises that have been made.  I am much more concerned that the system will not work well and create a lot of false alarms and miss threats often unless the more intrusive features of the system get turned on.  I fear the “Bait and Switch.” I fear that once this system is in-place at New Hanover County schools, we are going to be coerced or forced by the state government to continue to turn on more and more of the intrusive features of the system.  One has to ask why the General Assembly required us to select an AI System that has the capability of Facial Recognition, if they ultimately weren’t going to force us to turn it on down the road?

The Nation’s Guinea Pig

Eviden is a French company. It boasts of having its AI systems installed at airports and that they even provided security systems for the Olympics in Paris.  However, there are absolutely no K-12 school systems in the United States that currently use their AI system.  They do boast that they have their AI systems installed at some universities, but all of these universities are in foreign countries.  Our school district (along with Davidson County Schools) would be the guinea pigs for our entire nation.

And what do we get for efforts? In order for this system to actually provide value, a lot of user-acceptance testing work will need to be done on our end.  This will likely require more staff to handle the additional workload over the next couple of years.  This will cost our school district money that we could be using on hiring more teachers or on upgrading facilities.  The $3.2 million we would get from the state does not cover these additional costs.

Usually when someone participates in a drug trial, they not only get free medicine, but they also get paid compensation.  Where is our compensation?  It’s not like the General Assembly is offering us a $10 million dollar bonus for participating in this pilot program.

Conclusion:

In the Minority Report the Precrime Organization has a huge financial incentive to not be completely honest with the public.  Likewise, the Ed Tech and AI industries are multi-billion-dollar industries already, and the potential for public sector monetary gain is incredibly huge. Eviden has much to gain with this pilot.  If successful they would likely be granted an ongoing contract with the state to provide their AI systems to all of the 115 school districts in North Carolina.  Once they are established in our state, they might even be able to expand nationwide.  The stakes couldn’t be higher, and therefore we need to subject Eviden to the highest level of due diligence.  So far, it seems as though our school board Is not willing to make that kind of due diligence a priority.

Some state legislatures have created guardrails in the law for AI and crafted statutes that cover the ethical use of AI.  Our General Assembly has not.  It’s the Wild West just waiting to happen.  AI has the potential to provide great benefits to our schools and to our state if used properly.  During my 2024 campaign for the school board, I promised the public that I would stand tall for the rights of parents and students, and place that concern above all others.  I meant it! There are simply too many red flags with this AI Pilot Program. I will not risk unleashing “Big Brother” on the next few generations of NHCS students until the NC General Assembly creates the safeguards we need for AI. I am not sure if I have the votes on the school board to stop this pilot.  Perhaps I am only issuing a “Minority Report.”