Sensor Onboarding Troubleshooting Guide

This guide provides solutions for common issues encountered during the sensor onboarding process in CORIOLIX.

General Troubleshooting Approach

When troubleshooting sensor onboarding issues:

1. Identify the affected step - Determine which stage is failing
2. Check system logs - Review relevant log files for error messages
3. Verify prerequisites - Ensure previous steps completed successfully
4. Test components individually - Isolate the problem area
5. Document solutions - Record successful fixes for future reference

Step 1 Issues: Sensor Creation

Problem: Sensor ID Already Exists

Symptoms: - Error message: "Sensor ID already in use" - Cannot save new sensor record

Solutions: 1. Check existing sensors for duplicates:

SELECT sensor_id, sensor_name FROM sensors WHERE sensor_id = 'SENSOR_ID';

2. Modify the sensor ID to be unique 3. Follow the naming convention: 6 alphanumeric + last 6 digits of serial

Problem: Regex Parsing Failures

Symptoms: - "Verify Format" button shows errors - Parsed values don't match expected results - Sample data doesn't generate expected parameters

Solutions:

1. Common Regex Issues:

Issue	Problem	Solution
Missing escapes	Special chars not escaped	Use \\. for literal dots
Wrong quantifiers	+ vs * vs ?	Use + for one-or-more
Case sensitivity	Uppercase/lowercase mismatch	Use [Bb] for both cases
Whitespace	Spaces in data	Use \\s* for optional spaces

1. Test with simple patterns first:

   # Start simple
   ['^(?P<first_value>\\w+)']
   
   # Add complexity gradually
   ['^(?P<first_value>\\w+),\\s*(?P<second_value>\\d+\\.\\d+)']
   

2. Use online regex testers:

3. Test patterns with actual sensor data
4. Verify named capture groups work correctly

Problem: Required Field Validation

Symptoms: - Form won't submit - Missing field error messages

Solutions: - Review all required fields (marked with *) - Ensure dropdown selections are made - Verify numeric fields have valid numbers - Check that text fields aren't empty

Step 2 Issues: Parameter Configuration

Problem: Processing Symbol Errors

Symptoms: - Data transformation failures - Error: "Unknown processing symbol" - Parameters not appearing in processed data

Solutions:

1. For NoTransformation sensors:
2. Leave Processing Symbol field blank

3. Ensure transformation module is set to "NoTransformation"

4. For custom transformations:

5. Consult transformation module documentation
6. Verify symbol names match exactly (case-sensitive)
7. Check for typos in symbol names

Problem: Invalid Data Ranges

Symptoms: - Quality control flags all data as invalid - Warning: "Value outside expected range"

Solutions: - Review sensor specifications for realistic ranges - Consider environmental conditions at deployment location - Set ranges 10-20% beyond normal operating limits - Use conservative ranges initially, then refine based on real data

Step 3 Issues: Network Logger

Problem: Serial Port Access Denied

Symptoms: - Error: "Permission denied /dev/ttyUSB0" - Cannot read from serial port

Solutions: 1. Add user to dialout group:

sudo usermod -a -G dialout $USER
logout  # Log out and back in

1. Check port ownership:

   ls -la /dev/ttyUSB*
   

2. Set proper permissions:

   sudo chmod 666 /dev/ttyUSB0
   

Problem: No Serial Data Received

Symptoms: - Empty log files - No data in network transmission - Timeout errors

Solutions:

1. Verify physical connections:
2. Check RS-232 cable connections
3. Verify correct pin wiring (TX/RX, ground)

4. Test with different cable if available

5. Check serial settings:

   # Test with screen or minicom
   screen /dev/ttyUSB0 9600
   
   # Check for data flow
   cat /dev/ttyUSB0
   

6. Verify sensor configuration:

7. Confirm sensor is powered on
8. Check sensor output is enabled
9. Verify baud rate, data bits, stop bits, parity

Problem: Network Transmission Failures

Symptoms: - Local data visible but not transmitted - Connection refused errors - Network timeout errors

Solutions:

1. Check network connectivity:

   ping [destination_ip]
   telnet [destination_ip] [port]
   

2. Verify firewall settings:

   sudo iptables -L
   sudo ufw status
   

3. Check port availability:

   netstat -an | grep [port]
   lsof -i :[port]
   

Step 4 Issues: Data Logger

Problem: Database Connection Failures

Symptoms: - Error: "Can't connect to MySQL server" - Authentication failures - Connection timeout errors

Solutions:

1. Check database service:

   sudo systemctl status mysql
   sudo service mysql status
   

2. Verify credentials:

   mysql -u coriolix -p -h [host] [database]
   

3. Check database permissions:

   SHOW GRANTS FOR 'coriolix'@'%';
   

Problem: No Data in Database

Symptoms: - Network logger shows data transmission - Database queries return no results - No errors in logs

Solutions:

1. Check table structure:

   DESCRIBE sensor_data;
   SHOW TABLES LIKE '%sensor%';
   

2. Verify data format:

3. Check timestamp formats
4. Verify sensor_id matches exactly

5. Confirm parameter names are correct

6. Monitor real-time insertion:

   tail -f /var/log/mysql/general.log
   

Problem: Data Quality Issues

Symptoms: - Data values seem incorrect - All parameters showing same value - Inconsistent parameter parsing

Solutions:

1. Verify regex parsing:
2. Test message format string with actual data
3. Check for parameter order mismatches

4. Validate named capture groups

5. Check transformation modules:

6. Review calibration equations
7. Verify unit conversions
8. Test with known input values

System-Wide Issues

Problem: High System Load

Symptoms: - Slow response times - Process timeouts - System becoming unresponsive

Solutions:

1. Monitor system resources:

   top
   iotop
   free -h
   df -h
   

2. Check log file sizes:

   du -sh /var/log/*
   # Rotate or clean old logs if needed
   

3. Optimize configurations:

4. Reduce logging verbosity
5. Increase buffer sizes
6. Consider data sampling rates

Problem: Time Synchronization Issues

Symptoms: - Timestamps don't match reality - Data appears in wrong time slots - Synchronization errors

Solutions:

1. Check system time:

   date
   ntpstat
   chrony sources -v
   

2. Synchronize clocks:

   sudo ntpdate -s time.nist.gov
   sudo systemctl restart ntp
   

3. Configure automatic sync:

   sudo timedatectl set-ntp true
   

Diagnostic Commands

Quick System Check

#!/bin/bash
# Quick CORIOLIX diagnostic script

echo "=== System Status ==="
uptime
free -h
df -h

echo "=== OpenRVDAS Services ==="
systemctl status openrvdas-logger
systemctl status openrvdas-central

echo "=== Database Status ==="
systemctl status mysql
mysql -u coriolix -p[password] -e "SELECT COUNT(*) FROM coriolix_db.sensors;"

echo "=== Network Connectivity ==="
ping -c 3 [central_system_ip]

echo "=== Recent Logs ==="
tail -20 /var/log/openrvdas/*.log

Data Flow Verification

# Check complete data pipeline
echo "1. Serial data:"
timeout 5 cat /dev/ttyUSB0

echo "2. Network transmission:"
sudo tcpdump -i eth0 -c 5 port [configured_port]

echo "3. Database records:"
mysql -u coriolix -p -e "SELECT COUNT(*) FROM sensor_data WHERE timestamp > NOW() - INTERVAL 5 MINUTE;"

Getting Additional Help

Log File Locations

• OpenRVDAS logs: /var/log/openrvdas/
• System logs: /var/log/syslog
• MySQL logs: /var/log/mysql/
• Application logs: /var/log/coriolix/

Support Channels

1. Check documentation:
2. CORIOLIX User Guide
3. API Documentation

4. System Administration

5. Contact support:

6. Create issue in GitHub repository
7. Email CORIOLIX support team
8. Check community forums

Escalation Process

When issues persist:

1. Gather diagnostic information:
2. System specifications
3. Error messages
4. Log file excerpts
5. Configuration files

6. Timeline of events

7. Document troubleshooting steps:

8. What was tried
9. Results observed

10. Current system state

11. Prepare for support contact:

12. Sensor specifications
13. Network topology
14. Custom configurations
15. Business impact

Prevention

Best Practices

• Regular backups of configurations
• Monitor system resources proactively
• Test changes in development environment first
• Document all customizations
• Keep systems updated with security patches
• Regular maintenance schedules

Monitoring Setup

# Set up automated monitoring
crontab -e

# Add monitoring jobs
*/5 * * * * /scripts/check_sensor_data.sh
0 */6 * * * /scripts/system_health_check.sh
0 2 * * * /scripts/log_rotation.sh

This comprehensive troubleshooting guide should help resolve most issues encountered during sensor onboarding. Remember to document any new solutions for future reference.