A guide to Disaster Preparedness, our thoughts on the topic, and many helpful ideas and suggestions
Your mobile/paperless EMS operation is dependent on its internet connections and on its internet-based resources, particularly the AngelTrack cloud server. We require all EMS agencies to lay groundwork for operational continuity in the event of total loss of all internet resources, as might occur when:
- A billing mistake at your internet service provider temporarily cuts off all service to your stations and mobile devices.
- A pipeline explosion or similar disaster wipes out AngelTrack's primary data center (located at the internet backbone nexus in Dallas, Texas) and it takes hours or a whole day for AngelTrack staff to reestablish your cloud server at a different data center.
- Your station is burgled and all of your mobile devices and dispatch computers are stolen.
- A flood wipes out wireless and internet service in your town.
- Et cetera.
It could happen and you must be ready for it.
Keep Paper Forms Ready and Waiting
It is likely, at some point in your operation's history, everything was run on paper. Once upon a time each vehicle carried a stack of those forms, ready to be filled out.
Once you go mobile/paperless, you can keep those paper forms in the vehicles. You can purchase a plastic document box, fill it with every form anyone might need, label it with the truck's name, and keep it onboard at all times. (The onboard box of paper forms can even be a checklist item that the crew verifies each morning.)
Dispatchers can keep a folder of call-taking forms, such as a folder full of them mounted on the wall, ready at a moment's notice in case the station's internet connection goes down. All dispatchers should understand how the process of taking a call using the paper form.
Set Up Nightly Schedule Notices
Each night, AngelTrack will send you a copy of the next day's call schedule, if you configure your messaging settings to allow this.
In the event of total loss of internet or AngelTrack, that call schedule will be the one critical piece of data that your organization requires. We strongly recommend setting yourself up to get a copy of the next day's call schedule emailed to you every night.
It is easy to set up an inbox rule to automatically file those emails away in a folder, always ready in case they are ever needed.
Alternate Communication Channels
Maintain an alternate communications channel for communicating with your crews:
| Primary Communications Channel | Cheapest Secondary Channel | Next Cheapest Secondary Channel |
|---|---|---|
| Company-owned Nextel phones | Employee personal cellphones | VoIP software on the company-owned mobile devices |
| Company-owned pagers | Employee personal cellphones | VoIP software on the company-owned mobile devices |
| Employee personal cellphones | VoIP software on the company-owned mobile devices | Company-owned pagers |
| Shortwave radio | Employee personal cellphones | VoIP software on the company-owned mobile devices |
| VoIP software installed on company-owned mobile devices | Employee personal cellphones | Company-owned pagers |
More likely than not, you will have to fall back on that secondary channel... especially in built-up areas and inside hospitals, where one type of signal fails while other types succeed.
Also, remember that AngelTrack can be configured to email out all details of the dispatch to crew members -- all the information they need to run the call. These emails are sent when dispatch assigns the call, and they can be sent as text messages if your crew members have configured their AngelTrack messaging settings to come to their personal phones as text messages.
Secure the Dispatch Office
You must protect your dispatch office from loss of electrical power and from loss of internet service.
Short-term loss of electrical power
Protect your dispatch computers from short-term power outages by connecting all dispatchers' computers to UPS devices. Resist the temptation to connect more than one computer/monitor to each UPS; each dispatch computer should have its own. And you must include the monitor and the network switch (if present), since the computer isn't useful without them.
Likewise for the networking hardware in your wiring closet. All devices -- including your phone switch, network switch(es), internet gateway, cable/DSL modem, WiFi access point, and your VoIP gateway -- must be connected to a UPS.
New UPS units are expensive. If there is not room in the budget for new ones, consider buying refurbished units instead: as long as the battery has been replaced, it will work just as well as a new unit. This eBay search shows refurbished UPS units currently for sale, sized 1000VA in order to power multiple computers and their monitors.
Long-term loss of electrical power
A sustained power failure at your dispatch office will require a gas-powered generator, because UPS devices rarely last longer than an hour or so.
Standalone gas-powered generators are cheap these days -- perhaps $500 for something large enough to run just your dispatch computers and associated networking infrastructure. You may also have an ambulance with a sufficient on-board generator, which you could park at the dispatch office to provide power.
Don't forget electrical cords. You will need a large-gauge extension cord or two, long enough to run from the parking lot (where the generator or ambulance will be stationed) all the way to the dispatch office and all the way to the wiring closet. This is probably longer than you think, so perform a test deployment to make sure the cords will reach.
Keep the cords and the generator and spare fuel all together in one place, and do not allow anyone to borrow the cords or fuel for some other purpose. Train your station captain and your dispatchers how to deploy the system all by themselves.
Sometimes it is hard to get a generator to start, if it has not been run in a long time. Here is a checklist for getting it running:
- Check that the main power switch is turned on.
- Check that the fuel valve is turned on. It is a metal valve that you will rotate 90 degrees to open or close. It is probably located on the generator's left side, immediately below the fuel tank.
- Check that there is fuel in the tank.
- Check that the choke lever is in the closed (choked) position. Choke is used to start the engine; open (unchoke) the valve as soon as the engine starts.
- Check the oil level using the dipstick; many generators have an automatic low-oil shutoff.
- If it still will not start, then use a screwdriver to open the carburetor drain for a few seconds until fresh clear gasoline runs out. If the generator has been sitting for many months with its fuel valve open, the carburetor float bowl will be filled with thick gasoline fractions which cannot be carbureted. If nothing runs out even though the drain is open, then check the fuel valve.
Loss of internet service
If internet service to your dispatch office is interrupted, you can run your operation from a wireless device over the cellular network. You can even run it with just a smartphone. Or send your dispatchers home with orders to run the operation from their home internet connection.
This is normally easy -- in fact you may already have a night dispatcher working from home. If you have not, then do at least one test run, including forwarding the phones.
In any case, it is a good idea to keep at least one cellphone handy -- be it personal or company-owned -- that uses a completely different carrier than your normal mobile provider. That way, if your normal carrier goes down, you can tether all your dispatch computers to the cellphone on the alternate carrier.
Train Employees How to Handle a Total Communications Outage
You should have a clear protocol for how crews should behave during a total communications loss, and the protocol should be trained up ahead of time, before you need it. A typical protocol looks like this:
Protocol for Total Communications Outage
- Complete all dispatches currently assigned. If a member of the crew has their hands free, continue attempting to reestablish communications with dispatch.
- Remain at your last dispatch's destination for ten minutes while attempting to reestablish communications. If all else fails, borrow the land-line telephone at your destination and call the company main number.
- If still no luck, return to your last posting station (headquarters or otherwise), while continuing attempts to reestablish communications.
- Remain at your posting location until either 1) the time of shift end, or 2) an employee delivers a paper call schedule to you.
24-Hour Continuity of Service Requirement
We here at AngelTrack LLC spend a great deal of time, effort, and money to prevent such an outage from ever occurring. For example, we do not attempt to save money by using a cheap datacenter; instead we pay top dollar for Rackspace, the top-shelf cloud hosting provider. They are gratifyingly obsessed with uptime.
The goal is to prevent any disaster that can possibly be prevented. But some disasters cannot be prevented. A meteorite or a terrorist or a sinkhole could destroy the whole Rackspace datacenter. If that happens, then AngelTrack support staff will scramble to activate a replacement cloud server for you, at one of the surviving datacenters.
Of course we would do that as quickly as possible. The problem is, if an entire datacenter goes down, then all of the AngelTrack customers based at that datacenter must be restored at alternate datacenters. It could take many hours to do them all. So, to protect yourself, you must be prepared to run for up to 24 hours without AngelTrack.
This is not just a warning; you must do it
When you signed the Terms of Service document, you agreed to plan, implement, test, and guarantee that your EMS operation can tolerate a twenty-four (24) hour outage of AngelTrack service.
We are serious. You really have to do it.
We will not sell AngelTrack to you, unless you can live for at least 24 hours without it.
Until you have implemented your continuity of service plan, and proven your ability to run smoothly for 24 hours with no AngelTrack service, you must cease all use of the AngelTrack product as soon as practical or within 72 hours, whichever comes first.
Enforcement and testing
At least once every six months, test your fallback systems and verify that everyone knows their role and can find the paper forms and other resources they need. There are many players involved in delivering AngelTrack service over the internet, and delivering internet service to your employees... and most of those factors are beyond the control of AngelTrack LLC. So no matter how carefully your AngelTrack cloud server is managed, a prolonged service outage could happen at any time. YOU MUST BE READY FOR IT.
Catch-Up Bookkeeping After an Outage
If you run on paper for several hours or a day, it will be necessary to input those calls into AngelTrack and then back-date them. To learn how to do this, read the Retroactive Editing and Booking guide.