Introduction

Healthcare in developing countries often lags due to obsolete technology and a lack of resources. One such critical area is Intensive Care Units (ICU), where there's an urgent need to upgrade the system of data logging from monitors. Current systems depend heavily on manual inputs, leading to a significant waste of valuable nursing time and a higher chance of error. This post presents my journey of building a solution to this issue: the ICU Monitor OCR Recorder.

Identifying the Issue

Across developing nations, data from ICU monitors often go unrecorded and un-stored. The primary reasons for this include the presence of outdated ICU models from various vendors and each vendor's attempt to establish a closed-off ecosystem. Moreover, the cost of streaming and storing data in such systems can be exorbitantly high.

These factors contribute to a critical yet under-recognized problem. Nurses, already stretched thin in the developing world, spend up to a quarter of their shift manually recording data from each ICU monitor. This is a wasteful use of resources, considering that there's a direct correlation between nurse shortage and higher mortality.

Developing the Solution

To counter this issue, I focused on the one universal interface that all the devices have to have: The screen. I developed an Optical Character Recognition (OCR) tool to read and log the data from ICU (and other) monitors. This tool, (smartphone running a dedicated app), is positioned in front of the monitor to continuously record the screen. The user annotates the tag for each region of interest (Heart Rate, Respiratory Rate, Photoplethysmogram, Temperature, etc.), once the roi's are marked the user can walk away. From then on a frame is captured at 1Hz, each region of interest is cropped and sent to a custom locally running optical number recognition model. Significant improvement was achieved over tesserachtOCR, which was the best open source OCR solution at the time, by limiting the number of possible characters to 0-9 (and None) and by fixing the possible output length to 3, with one head per number. This meant that the model could only recognize numbers from 0 to 999, but it did so robustly with a (relatively) small backbone (Resnet- 50, I think) and a relatively small data set of < 1m images.

To speed up the process of prototyping and testing, I designed the first POC as a smartphone app, which I developed using Flutter which was the hot new thing at the time. I learned to use flutter specifically for this project, however I still really enjoy using it today.

Data generation

I was consumed by this project for a long time and there is a lot to talk about, but the bit that might be most interesting from a machine learning perspective (and telling about my personal style) is the data generation. I needed a fairly large set of annotated pictures of numbers. Ideally these pictures should contain all the artifacts that one should expect from the real world data. To do this I wrote two bits of software, one that generated a series of frames and shoved them into a video file, and one that decoded the same frames from the re-recorded video. Each set of number frames was preceded by a marker frame. A marker frame had a bunch of QR codes indicating the location and value of the subsequent numbers. This marker frame was followed by a set of frames that placed the expected numbers at the expected locations.

Testing the Solution

The POC was tested in a clinical setting, garnering positive feedback from both doctors and nurses.

Challenges and Future Work

In the pursuit of funding to elevate the project from POC to a full-scale company, I teamed up with the tech transfer office at my university. As the first step, they filed a patent on my behalf. However, this move ended up being a stumbling block for the project's development.

Fortunately, the patent was only filed in South Africa (South African Patent 2018/04931), so others outside the country are welcome to explore this technology. An excellent professor in the US is currently also pursuing this concept, and I can provide contact details on request.

Legislation

Medical certification is often cited as the thing that makes this project infeasible, however I disagree. I don't think that this system should be seen as a medical device. There is no transducer that probes something squishy and wet to come up with a number. It's far more similar to the document scanners that are already in wide spread use in hospitals and are not considered medical devices. Additionally consequences associated with miss classifications are similar to that of a document scanner, however because each classification is re-computed every second (and every sample comes with an associated certainty) it makes error rejection (and continuous improvement) much simpler.

The business models

I hold out hope for two approaches. In one approach it is sold directly to doctors (in it's app form) that will allow them to have a continuous data stream of all their patients, without needing to rely on the broken telephone which currently exists. In the other approach it is sold to the hospital as a system (central node with dedicated camera modules, think esp32CAM), and the hospital can then use it to improve the efficiency of their nurses. The first approach seems easier, but the second approach is more likely to have a large impact. A promising spin on the second approach is to partner with a EMR (Electronic Medical Record) or HIS (Health Information System) providers and sell the solution as an add-on to their existing system. The EMR/HIS market is a fiercely competitive one and no one seems to have a distinguishing feature like this, the first one to implement this will force the rest of the pack to follow suit. In the limit this hack might even be enough to force the big players to open up their systems, which would be a huge win for the healthcare sector in developing countries.

Conclusion

My ICU Monitor OCR Recorder was an attempt to bridge a gap in the healthcare sector of developing nations. It's a testament to how a simple hack combined with some machine learning fairy dust can enable improbable systems. While I have put down this torch for the time being, I think it's a really powerful idea, and if someone can make it work it will be a real kick in the pants to the powers that be. I encourage those interested in furthering this project to reach out and continue the journey.