The Problem with Monitoring

The first step towards proper AV monitoring is admitting The Problem: the tools are simply not there. Admit it. You are flying blind. You don’t even know what you have. Answering the simple questions of, “What is our total number of conference rooms? And how many are working right now?” in a larger organization is blurry at best. This is not an easy problem to solve, due to the isolated AV ecosystems and vendor allegiances.

And even if you can solve it, you may not yet be able to implement it, because your solution is not scalable. Or not manageable, or both. For now, your best solution is to include multiple windows to monitor the siloes. Human eyes have to jump from window to window or screen to screen, trying to piece together the data in real-time, which often leads to information overflow for the human mind.

Monitoring is a fundamental step that gathers the telemetry data from AV devices, automatically. This level of automation requires some level of programming skills to implement. It also assumes that the data you need is readily available, and that you can somehow check the data for integrity.  

At Level 3 Audiovisual, we are taking a more standardized, templated approach, more cattle, less pets.

Pets vs. Cattle

Pets vs. cattle is a cloud-scaling analogy that has emerged over the past 10 years; here is a summary:

“In the old way of doing things, we treat our servers like pets, for example Bob the mail server. If Bob goes down, it’s all hands on deck. The CEO can’t get his email and it’s the end of the world. In the new way, servers are numbered, like cattle in a herd. For example, www001 to www100. When one server goes down, it’s taken out back, shot, and replaced on the line. “

Source: Randy Bias, The History of Pets vs Cattle and How to Use the Analogy Properly, Cloudscaling

Seek Observability

With a proper nod to Schrodinger’s cat, what we are seeking from AV devices is general “observability.”; the ability to know everything about a device by listening to its outputs; not the typical audio output and video out signals, but device status, usage data, and downtime alerts. More on that later in the post.

Assuming the telemetry data is there to get, the first step is to configure one way of getting the data from a server (or an AV device).  For example, the Poly x30 and x50 models have the same API. 

Then we want to apply that ONE WAY of data collection (our template) to all devices in our WAN, keeping the number of templates to an absolute minimum, which drives standardization, but allows for custom rooms. Some AV folks might look at these templates much like we think of “drivers”, if that helps. If we need to customize the way one of our templates performs in a given environment, then we set up inheritance between templates, and override certain qualities of the template for certain devices.

In this future ideal world, all of our AV device would support SNMP (Simple Network Management Protocol) and other standard API devices. There would be proper alerts, along with machine learning, training the monitoring systems.  Dashboards would provide access to important outage data, network performance, and service needs. 

This is known as service monitoring in the IT world: a room is service, it has components, then you can present a service status, then you can present Service Level Agreements (SLA’s) to the management. Eventually, you can have SLA counting, carried across all rooms, this is all eventually, ideally... 

The truth is, we are not there, yet, but that is we at Level 3 Audiovisual are heading to in the near future.

AV consultants could bring more value to the table by requiring SNMP, telemetry, and general “observability” requirements of AV devices that they specify into their projects. If an AV device does not provide the proper telemetry data, giving your team everything they need to monitor a room system and usage, it’s out.  Of course, right now, many specified products often get replaced due to supply chain issues, creating often unforeseen cybersecurity risks. The camera is not available? Okay, so I found this other one on Amazon.

Ensure your data remains safe and secure

As mentioned earlier, what we are seeking is observability, but we also need to ensure the data we are getting remains safe and secure, ensuring data integrity, that our data is not being “rounded” or changed inadvertently, or maliciously. We need verifiable observability (see the Stuxnet virus for an example of what can happen if the data we are getting from the devices is not accurate or up to date).

We need secure, accurate data, from the AV devices, without opening up security holes along the way.

The goal is to provide the proper tools to troubleshoot, possibly repair, or at least remotely access, and/or resolve the issue.  Maybe a reboot is needed to as a temporary fix before it can be addressed during a scheduled downtime of the room?  Maybe a new display or other hardware is needed?

The Problem with Monitoring

In the end, we (ideally) know what AV devices you have, how each device interacts and depends on other devices. Isn’t that really what an AV diagram should be showing? Not just electricity, but logical connections, and data dependencies.  A codec can’t work without a working camera and microphone.

Proper monitoring depends on the proper data from the devices, and the proper auditing of that data depend on proper monitoring. Monitoring is needed for audits, and audit logs are part of monitoring. “  “Monitoring is a type of watching or oversight, whereas auditing is recording of the information into a record or file. It is possible to monitor without auditing, but you can’t audit without proper monitoring.” – from Chapple, Stewart and Gibson, CISSP Certified Information Systems Security Professional Official Study Guide.

You can’t audit without proper monitoring. You can’t monitor without proper tools, and tools also depend on secure data, which is not always available. This is The Problem or Problems with monitoring. 

WANT TO LEARN MORE ABOUT CYBER SECURITY IN AV?

Continue reading ​Attack Vector or get connected with us! You can contact us here or click the chat box below to connect instantly. We look forward to chatting with you!