Sensors - L1 Troubleshooting

Introduction

A majority of sensor-related issues are either power or network connectivity related, and the intent of L1 troubleshooting is to check both of these and fix any problems that are found. To check power and network, you need physical access to the store infrastructure, so you need to be physically at the store or be in communication with a technician or store personnel who is at the store.

L1 troubleshooting is generally handled within the customers' IT organization as the intimate knowledge of the store environment helps to resolve issues faster and more economically. In cases where L1 troubleshooting does not fix the problem, the issue should be escalated to RetailNext Support for more advanced L2 troubleshooting.

Preparing

Check for known issues at the store

If possible, try to find out if there are known power or network issues at the store before you visit it.

This information may have been already reported by store personnel and recorded in an internal IT/ticketing system, or you can make a quick call to the store to check with personnel on whether they are facing any power or network issues at the moment.

Collect information

Gather information on as much of the following as possible prior to your visit. If you received a health monitoring alert for the sensor issue from the RetailNext system, the alert will contain information such as the store name, sensor name, and sensor IP. It will also contain a link to the sensor's page on the RetailNext Cloud UI's Camera Admin section from where you can obtain more details if needed. See https://retailnext.atlassian.net/wiki/spaces/KB/pages/145293496 for more information about alerts.

1. RetailNext cloud subscription information

   • Subscription URL i.e. https://acme.cloud.retailnext.net

   • Login account

2. Details about the specific sensor(s) that are having issues

   • Sensor model

   • Sensor IP address

   • Sensor MAC address

   • Sensor username and password

   • Sensor install height

   • Issue type (sensor down? rebooting frequently? something else?)

3. Store network setup

   • Static IP or DHCP configuration

   • DHCP server IP address, if using DHCP

   • Gateway and DNS server IP addresses, if using static IP

   • NTP server IP address

4. Store power supply (PoE) setup

   • PoE device type (switch? injector?)

   • PoE device model

Ensure that you have the necessary tools

1. Laptop (Windows, Mac, or Linux)

2. The RetailNext https://retailnext.atlassian.net/wiki/spaces/KB/pages/189399050(http://netutil.retailnext.net) installed on your laptop

3. Ethernet cable tester

4. Ethernet (Cat5e) cable

5. PoE injector

6. Screwdrivers

7. Ladder

Step-by-step Instructions

Step 1 - Check the sensor status LEDs

The sensors have an LED that flashes in a specific pattern to indicate the status of the sensor. This allows for quick visual identification of issues.

Once you are at the store, locate the sensor and examine the pattern of its status LEDs. The status LED on an Aurora is on the side of the sensor, and on a Xovis it's on the front.

Aurora

• LED not blinking at all: Sensor is likely not receiving any power. Proceed to the "PoE power" section below.

• LED blinking Green every 30 seconds: Sensor is working ok.

• LED blinking Blue, then Green (B G): Sensor is unable to obtain an IP address and/or the DNS server's address from the DHCP server. Proceed to the LAN section below.

• LED blinking Blue, then Green twice (B G G): Sensor has a certificate error and needs to be replaced.

• LED blinking Blue, then Green thrice (B G G G): Sensor is unable to connect to the internet. Proceed to the WAN section below.

For more information, refer to the page https://retailnext.atlassian.net/wiki/spaces/KB/pages/145031178

Brickstream 3D (2300)

Brickstreams have three LEDs. For the purposes of troubleshooting, examine only the center LED.

• LED blinking Amber every 3 seconds: Sensor has an issue with DHCP, DNS, or NTP. Proceed to the LAN section below.

• LED blinking Green every 3 seconds: DHCP, DNS, and NTP are ok. Data push may or may not be ok. Proceed to the WAN section below.

Note again that the center LED flashing Green does not mean that the sensor is fully operational; it simply means that there is no issue with DHCP, DNS, or NTP. If there is an issue with the WAN that is causing the sensor's data push to the cloud to fail, this LED will still flash Green.

Brickstream 3D Gen2

The 3D Gen2 sensors have 2 LEDs, whose light functions are detailed below.

Note: The LED lights may turn off after booting up if they are configured to do so in the setting “Defining the Device Identification.”

Open

• LED 1:

  • Blinking amber at three-second intervals: Connected with neither the time sync not the data server

  • Blinking green at three-second intervals: Connected to time sync and/or data servers

• LED 2:

  • Red: Operating with factory default IP address (192.168.1.7)

  • Progressive:

    • Indicates whether the sensor has reverted back to its default factory settings

    • Also used with the manual reset button when resetting the sensor

Contact Support if you see a different LED combination to what is documented here.

Xovis

• LED not blinking at all: Sensor is likely not receiving any power. Proceed to the "PoE power" section below.

• LED blinking Orange, then Green (O G): Sensor is working ok.

Note that if the Xovis sensor is white in color, then that means it has a white plastic cover that hides the status LED. To examine the LED, either remove the cover, or if the sensor is not easily physically accessible then skip this step and proceed to the other troubleshooting steps below.

Step 2 - Check the sensor's local webpage

Ping the sensor's IP address from your laptop and ensure that it responds. You can obtain the sensor's IP address from its Camera Admin page in the RetailNext Cloud UI. For Aurora sensors, you can also obtain the login information from the admin page.

Open the sensor's local webpage on your browser (http://<sensor ip>).

Aurora

Log in using the credentials on the Camera admin page in the RetailNext Cloud UI. After logging in, look for any errors displayed on the camera's webpage.

Open

Brickstream 3D 2200/2300

Log in using the credentials used for Brickstream sensors. After logging in, click on the "Diagnostics" link at the top. In the "1. Statistics" sub-page, locate the "Tallies" table and look at the "Last Failure" column to see if the NTP time sync, Device manager connection, or Batch data push to the cloud has failed recently. Go to the "2. Log" sub-page to view details on any failures.

Brickstream 3D Gen2

Log in using the credentials used for Brickstream sensors. After logging in, examine the “Status Overview” and “Recent Log Messages” sections to check for any recent failures or error messages. Click on “View Detailed Logs” at the bottom to see additional details about the log messages.

Xovis

Log in using the credentials used for Xovis sensors. After logging in, click on the "Status" link at the top and look for any errors displayed on this page.

Step 3 - Check the store network (LAN)

Use an ethernet cable to connect your laptop to the same switch that the sensor is plugged into. If the sensor is connected to a PoE switch, plug your laptop into the same switch. If there are no free ports on the switch or the sensor is connected to a PoE injector, plug your laptop into the same switch or router that the PoE switch/injector is connected to.

DHCP configuration

If the sensors are set up for DHCP, then ensure that your laptop receives a DHCP address from the store DHCP server. You can use the "ipconfig" tool on Windows, and the "ifconfig" tool on Mac/Linux to confirm this.

Static IP configuration

If the sensors are set up for static IP, then configure your laptop's ethernet interface to use a static IP in the same subnet as the sensors, and with the same Gateway and DNS server IPs that the sensors were configured with. Ensure that you are able to successfully ping both the Gateway and DNS server IP addresses.

Step 4 - Check the store internet (WAN)

Use the RetailNext https://retailnext.atlassian.net/wiki/spaces/KB/pages/189399050 (netutil) to ensure that the store internet connection and firewall rules are configured as expected. netutil also checks the DHCP and DNS configuration on the network.

Run the tool on your laptop as instructed. Select the sensor type and the network type and then run the test. If any problems are reported, fix them and then rerun the tool to ensure that the problems have been fixed.

Step 5 - Power cycle the sensor

Next, try power cycling the sensor by unplugging its cable from the PoE switch/injector, waiting a few seconds, and then plugging it back in.

Step 6 - Check PoE power

All sensors are powered by PoE (Power over Ethernet). Locate the PoE device used to power the problematic sensor. This can either be a PoE switch that can power multiple sensors or an injector that powers a single sensor.

Confirm that the sensor is plugged into the PoE switch/injector. Next, confirm using the LED indicator on the PoE switch/injector that it is receiving power. 

If you brought a spare PoE injector, swap the current PoE switch/injector with your spare one. Follow these steps:

• Plug your spare PoE injector into a power socket

• Unplug the sensor from the PoE switch or injector it is currently connected to, and instead connect it to the "PoE" port on your spare PoE injector

• Connect the "LAN" port on your spare PoE injector to whatever network device the current PoE switch or injector is connected to. For example, if the current injector is connected to another switch or router, connect your PoE injector to that same switch or router

• Wait for the sensor to boot up and check if the issue gets resolved

If using your spare injector resolves the issue, then there is a problem with the current PoE device. If it is an injector, the injector needs to be replaced. If it is a PoE switch, check if the switch is getting overloaded due to too many PoE devices. If it isn't overloaded, then the switch needs to be replaced.

If the current PoE switch/injector is working fine, proceed to the next troubleshooting step.

Step 7 - Check the ethernet cable

As a final step, check if the ethernet cable or terminations are faulty.

First, test the cable using your cable tester. If it passes, then temporarily remove the sensor from the ceiling and "short-cable" it by connecting it to the PoE switch/injector using the short ethernet cable you brought with you. See if this fixes the problem.

Summary of Steps

Below is a summary of the troubleshooting steps, in order:

1. Check the sensor status LEDs. See if the pattern lets you quickly identify the issue.

2. Check the sensor's local webpage. Check if any errors are displayed on this page.

3. Check the store LAN using your laptop. Notify the retail IT team if there is a problem.

4. Check the store internet/WAN. Notify the retail IT team if there is a problem.

5. Power cycle the sensor. Check if this fixes the issue.

6. Check the PoE device. See if using a spare PoE injector fixes the issue.

7. Check the ethernet cable. Test it using a cable tester. "Short-cable" the sensor using a different ethernet cable. Replace the current cable if the current cable is faulty.