BLOOMINGTON, Ill. (KXAN) — Sitting on the outskirts of a midwestern city some thousand miles north of Austin, billions of student emails, homework assignments, instant messages and other communications are being screened around the clock.
The artificial intelligence on computers housed inside a red brick building on the outskirts of Bloomington, Illinois, reviewed nearly four billion documents last school year.
The technology searches for keywords and phrases on school software and electronics that could potentially be cause for concern. It does not review social media.
Of those four billion screenings, the AI flagged 70 million for human review.
When that happens, one of the 25 members that make up a security team at a company called Gaggle checks out the document.
“It will get escalated to a person who will make a decision on the level of severity,” said Gaggle’s Operations Vice President Heather Durkac. “From there it puts the plan in action to reach out to the district.”
Gaggle currently contracts with 1,300 districts nationwide, including 87 in Texas.
Coast to coast, the company screens the data of roughly 4.5 million students.
How it works in RRISD
Round Rock ISD is the company’s largest client in Central Texas.
“I get alerts on my computer every day, anywhere from 10 to 15 every day,” said RRISD Associate Director of Administrative Projects Gordon Perez.
Perez oversees the Gaggle partnership for RRISD, a district with more than 50,000 students.
He’s among a select group of administrators that receives communication directly from the company’s Bloomington office.
“The phone calls could be any time, during the night, and they have been,” Perez said. “When they say check it out, usually it’s going to be somebody who’s going to plan to hurt themselves, or somebody is planning to hurt others.”
Last year Gaggle alerted RRISD to 8,757 dangerous behaviors.
While the vast majority, 6,725, were for profanity, the company also flagged 424 references to violence toward others, 496 instances of nudity and sexual content and 637 mentions of suicide or self -harm.
Just as concerning is the increasing frequency with which students are making their outcries.
Alerting to a growing trend of self-harm
KXAN analyzed the district’s data dating back to 2013 and found since then, alerts for profanity are up 332%. Violence is up 489%. Nudity and sexual content grew 687%. Yet, more than any other category, suicide and self-harm is up 1,454%.
Perez credits the technology with saving Round Rock students’ lives.
He highlighted one instance where administrators called 911 directly after getting a call from Gaggle.
He said a Gaggle representative called him at home and said it appeared a student was on the verge of taking his own life.
“Because of our intervention, that particular student did not go forward and carry out his plan to hurt himself, and kill himself, actually,” Perez said.
Gaggle CEO Jeff Patterson said he hears stories like that all the time.
While visiting Austin for an education conference, KXAN met Patterson in a downtown hotel lobby where he explained the impact his company is having.
“Last school year we identified more than 700 students that we called ‘lives saved’ because the item that we found had such clear plans to kill themselves or somebody else, and it usually involved the means as well as a timeline,” Patterson said.
Those 722 students saved are just a sliver of a larger trend.
Within the 1,300 districts it screens, Gaggle flagged 52,000 references to suicide or self-harm in students’ online activity last school year. Of these, more than 6,000 were serious enough to merit immediate attention by the district.
‘Part of the solution’
Despite the technology’s life-saving potential, Patterson is the first to admit it isn’t a blanket solution to mental health.
“I think it’s part of the solution, so the piece that we currently provide is identifying these kids who are making poor choices, who are feeling depressed or in crisis right, trying to identify these kids before something tragic happens, but there’s a huge need to sort of help these kids once they’ve been identified,” Patterson said.
Back in Bloomington, Durkac compared the safety Gaggle provides to protections built-in on a playground.
“We cannot put students in a digital playground without that protection,” Durkac said. “Gaggle brings that protection for our customers.”
As a reminder of their work’s importance, Gaggle’s walls are covered in stars, each representing lives saves.
There were 722 stars on the wall last school year.
This year there are already 222.
