Testing day. Students are seated, the room is quiet, and the app will not launch. Or it launches and then a screen sits blank because something behind it will not load.
The confusing part is that nobody on your team changed anything since the last window. That is the reason it broke.
Between testing cycles the vendor changes the app and the URL list. Both move on their schedule, not yours.
The version on the devices is not the version you pushed
You deployed the testing app months ago. The push reported success. Since then the vendor has released a newer build, and the secure testing mode in most of these platforms refuses to run on an old one.
Your deployment record is not evidence of what is on the devices today. Between then and now, machines got reimaged, replacements arrived mid year and were enrolled from an older image, and a cart came back from repair with whatever was on it.
So pull a report of the installed version across the fleet and compare it to what the vendor requires this cycle. Not what it required when you deployed. The two questions look the same and they are not.
The URL list changes, and your notes will not tell you
Every vendor publishes a technical guide for each testing cycle. The temptation is to work from the notes somebody on your team wrote last year, because the process really has not changed.
The process has not changed. The URL list has. Vendors add domains, move services to new hostnames, and introduce a new endpoint for a feature that did not exist last spring. None of that appears in your notes, because your notes were accurate when they were written.
Jason Koch, vCIO at Dynamic Network Solutions, puts the whole thing in one line.
“Follow the current guide every cycle, even when you are certain nothing has changed,” he says. “The parts that change are the parts nobody thinks to check, and you find out on the day with a room full of students waiting.”
Allowlisting is two systems, and people fix one
This is where a careful team still gets caught.
Content filtering and pop up blocking are separate controls. A domain you have allowed in the filter can still be stopped by a pop up rule, and a pop up exception does nothing if the filter is blocking the domain outright. Both lists need this cycle’s URLs, and fixing one does not tell you the other is wrong.
There is usually more than one place each list lives. The filter has one. The browser policy pushed through your management console has another. Depending on how the network is built, the firewall may have a third. A URL allowed in two of the three fails in exactly the same way as a URL allowed in none.
Do it early enough to fix what you find
The checks above are quick. Acting on what they find is not.
A fleet wide app update needs devices powered on and connected, which for a cart in a storeroom means somebody wheels it out and plugs it in. A filtering change may need a vendor ticket. Neither of those happens the morning of the test.
Do the checks far enough ahead that you can still act on what they find. Finding the problem the day before is the same as not finding it.
The only check that counts
Everything above is preparation for one test, and it is worth doing properly.
Take a student device, on the school network, in the room where testing happens, and launch the app the way a student would. Get all the way to the screen where the test would begin.
Not a staff laptop, which sits on a different network segment with a different filtering policy. Not your desk on the admin VLAN. A real device, in the real room, on the real network.
If that works, the rest of your fleet is very likely fine. If it does not, you have found the problem while there is still time to fix it, which was the entire point.
Where we come in
Most schools we work with run this as part of a testing readiness check a few weeks before each window: verify versions across the fleet, re-read the current guide, reconcile the allowlists across every system that has one, then test end to end on a real device in a real room.
It is unglamorous work, and it is why testing weeks pass without incident. If you would rather not work through it alone, that is part of what we do, and we are happy to talk it through whether or not you are a client.