Michigan schools surveil student emails for suicide risks, raising ACLU concerns

Thirty-one Michigan school districts are using a controversial surveillance service that reads students' school emails to identify students in crisis and alert school officials.

The Michigan schools are among roughly 1,400 districts across the United States that have hired Gaggle, an Illinois-based company, to use artificial intelligence and staff to search nearly 4 billion emails and school-email based documents last school year for examples of bullying, inappropriate behaviors, school violence and other harmful situations inside students' communications.

The practice has raised concerns among civil libertarians. But the company claims to have prevented 722 suicides last school year based on feedback from school officials on specific cases.

"That number is probably much higher," Gaggle spokesman Bill McCullough said. "This is when the school told us we saved a life that day. Kids put suicide notes online. They will share with friends 'I am going to kill myself today.'" 

McCullough said the company does not track the number of prevented suicides per state and could not provide a specific figure for Michigan.

The company uses "machine-learning" algorithms to do a first pass over the data. Anything that is flagged is sent to a safety team for human review before it is sent on to schools, Gaggle officials said.

Gaggle has found emails in which a student is reaching out to someone else about suicidal thoughts or plans, McCullough said.

"We see those and we notify the district immediately. They send someone to do a wellness check on a kid at home or in school, "he said. "Many times they are in the middle of an attempt. We have stopped a suicide on school property."

Civil libertarians argue Gaggle's service amounts to invasive surveillance and the company cannot take credit for saving lives.

"We have numerous concerns about the idea that school districts are monitoring the private conversations and thoughts of all of their students," said Chad A. Marlow, a senior advocacy and policy counsel with American Civil Liberties Union. 

"Having adults intervene could actually drive them to self-harm. ... If they are taking credit for saving lives, they cannot make those claims. They cannot be scientifically backed up," Marlow said.

McCullough said the company does not examine students' social media posts because that is considered private information. But school accounts are operated by districts, and students are told they have no expectation of privacy on those platforms, he said.

"We are only protecting school-issued tools," he said. "We don’t look at private email addresses. That would be like walking into a kid's bedroom. They have an expectation of privacy in that area."

Public school districts are in charge of disclosing to students and faculty if they are using school issue accounts and devices, so they have no reasonable expectation of privacy, Gaggle officials said.

The company says its policy is to comply with all applicable state and federal laws when it has access to student data.

Safety drives Gaggle use

Schools in Michigan are constantly evaluating their approach to safety and security, from constructing new buildings to thwart school shooters to rethinking how lockdown drills are conducted for young students to considering arming school personnel.

Some school officials hope the service can help them address the issue of teen suicide in Michigan. Suicide is the second leading cause of death for young people in the state between ages 10 and 24, behind accidents.

Students are engaging in dangerous behavior from self-harm to violence toward others to the sharing of nudity and sexual content, Gaggle officials said, and they are leaving evidence in their online activity on such software as Google’s G Suite, Microsoft Office 365 and the Canvas learning platform.

Gaggle officials said the company does use personally identifiable information in students’ education records to perform its services. But unless a school official expressly instructs otherwise, the company will not share or reuse personally identifiable information from education records for any other purpose. 

Gaggle officials also declined to provide a full list of the 31 districts in Michigan using the service, but said one of its earliest users is the Grand Rapids district. Three Metro Detroit school districts are piloting Gaggle's service this school year, but the company would not identify the districts.

Michigan's experience

The Grand Rapids district has been using Gaggle for the last seven years.

Larry Johnson, executive director of public safety for the district, said he believes the Gaggle service has saved at least two lives among the 15,000 students in the West Michigan district. He said he hasn't heard a complaint from the public about students' privacy being infringed on.

"We absolutely had some opportunities to get some resources to parents and families in regards to students who may have put themselves in a self-harm situation," Johnson said.

"... I believe in civil rights. I also believe in the right to keep students safe. Those two families have their kids today."

During the 2018-19 school year, Gaggle analyzed and viewed more than 3.9 billion items of which 70 million items were reviewed for suspicious content, company officials said.

More than 100,000 "safety issues" were reported nationally among the nearly 5 million students under Gaggle monitoring in the 2018-19 school year. That turned into nearly 13,000 cases deemed serious enough to warrant immediate action, McCullough said.

The 2,300-student Eaton Rapids School District also uses Gaggle. The district gets "a handful of notices" from Gaggle every day to review, Superintendent Bill DeFrance said. Last school year, school counselors used the information to reach out to 18 students and offer help.

"It’s a shrink wrap around the kids' emails," DeFrance said. "The emails are school emails on our platform. I just saw one today. They are looking for sex, nudity, violence harassment, suicide. Those would get flagged."

Parents are informed annually of the service and that their child's email is being monitored, but the district does not remind the students, he said. The district does not hide it is monitoring emails and the kids talk about it after something or someone has been flagged, he said.

Calls from Gaggle go to the student's principal and counselor first and then higher up in administration if warranted, DeFrance said. The service is monitoring 24 hours a day. DeFrance said he once got a call on a Sunday from Gaggle about a student who may have been in trouble.

"I think it's a marvelous service. We've had no push-back from the community," DeFrance said. "I think it's caring. It’s not overstepping."

The district spends about $15,000 on the service.

"You can't put a price on a kid's life. If we've saved one kid, it's worthwhile," DeFrance said.

State officials give advice

During the 2018-19 school year, Gaggle found 52,000 references to suicide or self-harm in students’ online activity. Of these, more than 6,000 were reported to districts. 

McCullough said some educators think kids only communicate via social media or private email accounts. 

"They say they would never use school-issued tools. They put the best of the themselves on social media," McCullough said. "But they put their most authentic self on their school tools, using it as a diary. You see them open a document and just journal their innermost self."

State education officials said contracting with Gaggle is a local decision by districts, but schools should be aware they have to follow the law on student privacy.

"Gaggle (or any other vendor) would be responsible for adhering to any/all applicable local, state or federal statutes or regulations as it relates to student identifiable information and protection of said information," Michigan Department of Education spokesman Martin Ackley said.

"Each local entity should have board policy and administrative guidelines in relation to protecting student data, which would then be part of any vendor agreement."

While the Gaggle service offers many positives — spotting suspicious content that might help prevent student self-harm or gun violence — the question needs to be asked if such surveillance is worth the cost, said Diana Graber, author of "Raising Humans in a Digital World." 

"After all, Gaggle is tracking everything a student does online. Giving away that much personal information is a big cost," Graber said.

Every service a student uses online — such as Instagram, Snapchat and TikTok — collects and uses their personal information, such as names, passwords, contacts and content including photos and videos, she said. Even Google and Microsoft services, which most schools use, collect student personal information, she said. 

"So in that sense, all of these services are invasive to students," Graber said.

"This is a very important lesson for students to learn as they become digital citizens who will hopefully make wise choices about what personal information they share and understand how it is used."

jchambers@detroitnews.com