Telemetry & Physics¶
The Foundation of Digital Trust¶
In the Kinetic Trust Protocol, trust is not declared—it is derived. Every millisecond, billions of telemetry events flow through digital systems. The challenge is
The Data Compass¶
| Layer | What | Why | Action |
|---|---|---|---|
| Signals | Facts | Patterns | Alerts |
| Meaning | Context | Insights | Recommendations |
| Wisdom | Principles | Strategies | Decisions |
Use this grid to ensure telemetry covers all three layers: Facts (micro), Context (meso), and Principles (macro). Without all three, you can raise alerts, but you lose adaptive insight.
The Principle
"You cannot trust what you cannot measure. You cannot measure what you cannot observe."
This page details the telemetry architecture that feeds the KTP model, from individual packets to the Experience Score.
The Three Layers of Observation¶
We categorize telemetry into three distinct layers, each corresponding to a scale of observation in the Digital Physics model:
%%{init: {'themeVariables': {'fontSize': '18px'}}}%%
flowchart TB
subgraph L1["🔬 Layer 1: MICRO"]
direction TB
M1[High Volume]
M2[Low Context]
M3[Individual Events]
end
subgraph L2["⚗️ Layer 2: MESO"]
direction TB
S1[Medium Volume]
S2[Medium Context]
S3[Statistical Aggregates]
end
subgraph L3["🌌 Layer 3: MACRO"]
direction TB
A1[Low Volume]
A2[High Context]
A3[Strategic Metrics]
end
L1 --> L2 --> L3
style L1 fill:#1a1a2e,stroke:#87CEEB
style L2 fill:#16213e,stroke:#87CEEB
style L3 fill:#0f3460,stroke:#87CEEB
| Layer | Physics Analogy | Data Characteristic | Update Frequency |
|---|---|---|---|
| Micro | Quantum particles | Individual events, high entropy | Milliseconds |
| Meso | Thermodynamics | Statistical emergence, patterns | Seconds-Minutes |
| Macro | Celestial mechanics | Gravitational force, stability | Minutes-Hours |
Layer 1: Micro Telemetry¶
Physics Analogy
Particles in motion—individual, discrete events that have no meaning in isolation but form the foundation of all higher-order understanding.
Raw Packets¶
The fundamental particles of the network. Packets are the photons of the digital universe—discrete quanta of information that travel at finite speed and can be absorbed, reflected, or lost in transit.
| Field | Type | Description |
|---|---|---|
timestamp |
datetime | Capture time (nanosecond precision) |
src_ip |
string | Source IP address |
dst_ip |
string | Destination IP address |
protocol |
enum | TCP, UDP, ICMP, etc. |
port |
int | Destination port |
payload_len |
int | Payload size in bytes |
tcp_flags |
array | SYN, ACK, FIN, RST, etc. |
ttl |
int | Time-to-live hops |
| Metric | Calculation | Unit | Impact on ARQ |
|---|---|---|---|
| Throughput | Σ payload_len / time |
Gbps | Quality ↑ |
| Packet Loss | lost / total × 100 |
% | Quality ↓↓ |
| Jitter | stddev(inter_arrival_time) |
ms | Quality ↓ |
| Latency | response_time - request_time |
ms | Accessibility ↓ |
Logs & Events¶
The semantic layer of telemetry. Unlike packets, logs contain structured or unstructured text that describes what is happening within applications and systems.
Physics Analogy
Logs are the thermodynamic state variables—they describe the internal configuration and energy distribution of the digital machinery.
| Level | Description | Example |
|---|---|---|
DEBUG |
Detailed diagnostic | "Cache lookup: key=user_123, hit=true" |
INFO |
Normal operations | "Request processed in 45ms" |
WARN |
Potential issues | "Connection pool at 80% capacity" |
ERROR |
Failures | "Database connection timeout after 30s" |
FATAL |
Critical failures | "Out of memory, process terminating" |
| Metric | Calculation | Threshold | Impact |
|---|---|---|---|
| Log Volume | Σ bytes / hour |
Baseline ± 2σ | Anomaly detection |
| Error Rate | errors / total × 100 |
< 0.1% | Retainability ↓↓ |
| Unique Sources | count_distinct(source) |
Expected range | Coverage validation |
| Event Clusters | Temporal pattern analysis | N/A | Root cause analysis |
-- Splunk: Error rate trend with correlation to E-score
index=application level=ERROR
| bucket _time span=5m
| stats count as errors by _time, service
| join type=left _time [
search index=ktp_metrics metric=e_score
| bucket _time span=5m
| stats avg(value) as e_score by _time
]
| eval correlation = if(errors > 10 AND e_score < 80, "HIGH", "LOW")
Real-time Metrics¶
Field measurements of operational health—the temperature, pressure, and electromagnetic field strength of the digital system. Below are the Real-time Metrics (348 Signals) used in the math mechanics of KTP. Each tab groups signals by layer and includes the calculation model used for scoring and aggregation.
Notation
count(x)is event count per window, describing raw activity volume.rate(x)is count per second, capturing velocity and churn.pXX(x)is a percentile, used for tail behavior and outliers.uniq(x)is distinct count, used for cardinality and spread.z(x)is a z-score against baseline, used for anomaly detection.ema(x)is exponential moving average, used for smoothing.- All metrics are normalized to 0–1 before projection into the tensor.
| Metric | Expression | Example |
|---|---|---|
| Throughput_sps | sessions / second |
|
| Trust_Mass | 100 - Accumulated_Risk |
|
| Env_Friction_Risk | current_risk_score (0-100) |
|
| Accumulated_Risk | sum(risk_score) over session |
|
| Phase | enum(AUTH, RECON, LATERAL, ESCALATE, EXFIL, CLEANUP) |
|
| Time_Tplus | simulation_time_cursor |
|
| Event_Risk_Level | severity_enum(0,1,2) |
|
| Zone | categorical(network_zone) |
| Metric | Expression | Example |
|---|---|---|
| user_id | identity_key |
|
| Auth_Volume_by_User | count(auth) / window |
|
| Failed_Login_Rate | count(failed_login) / count(login) |
|
| Concurrent_Sessions | active_sessions_per_user |
|
| New_Device_Access | count(new_device_login) / count(login) |
|
| role | rbac_role |
|
| Privilege_Escalation | count(priv_escalation) / window |
|
| Role_Change_Frequency | count(role_change) / window |
|
| Toxic_Combination | count(toxic_pairings) / window |
|
| Dormant_Role_Usage | count(dormant_role_use) / window |
|
| department | org_unit |
|
| Cross_Dept_Access | count(cross_dept_access) / window |
|
| Dept_Outlier_Analysis | z(access_volume_by_dept) |
|
| Shadow_IT_by_Dept | count(unapproved_apps_by_dept) / window |
|
| geo_location | geo_coord |
|
| Impossible_Travel | count(impossible_travel) / window |
|
| New_Country_Access | count(new_country_access) / window |
|
| Geo_Velocity_Anomaly | z(geo_velocity) |
|
| High_Risk_Country | count(risky_geo_access) / window |
|
| device_id | device_key |
|
| Device_Trust_Score | ema(device_trust) |
|
| New_Device_Rate | count(new_device) / count(auth) |
|
| Jailbroken_Device | count(jailbroken_device) / count(device) |
|
| BYOD_Usage | count(byod) / count(device) |
| Metric | Expression | Example |
|---|---|---|
| http_method | enum(GET, POST, PUT, DELETE) |
|
| Method_Distribution | distribution(http_method) |
|
| Unusual_Method_Usage | z(method_rate) |
|
| High_Volume_POST | rate(POST) > baseline |
|
| Method_vs_Path_Anomaly | z(method_path_pair) |
|
| http_status | enum(200,403,500) |
|
| Error_Rate_5xx | count(5xx) / count(request) |
|
| Access_Denied_403 | count(403) / count(request) |
|
| Not_Found_Spike_404 | z(count(404)) |
|
| Success_Rate | count(200) / count(request) |
|
| url_path | path_key |
|
| Path_Traversal_Attempt | count(path_traversal) / window |
|
| Admin_Page_Access | count(admin_path) / window |
|
| Sensitive_File_Access | count(sensitive_path) / window |
|
| High_Cardinality_Paths | uniq(url_path) / window |
|
| user_agent | ua_string |
|
| Rare_User_Agent | z(ua_rarity) |
|
| Bot_Scraper_Detection | score(bot_score) |
|
| Outdated_Browser | count(outdated_browser) / window |
|
| UA_Spoofing | count(ua_mismatch) / window |
|
| referer | referer_url |
|
| Empty_Referer | count(empty_referer) / window |
|
| Cross_Site_Scripting | count(xss_indicators) / window |
|
| External_Referer | count(external_referer) / window |
|
| cookie_id | cookie_key |
|
| Cookie_Replay | count(cookie_replay) / window |
|
| Cookie_Theft | count(cookie_theft) / window |
|
| Missing_Secure_Flag | count(missing_secure) / window |
|
| Session_Fixation | count(session_fixation) / window |
| Metric | Expression | Example |
|---|---|---|
| api_endpoint | endpoint_path |
|
| Endpoint_Usage | count(endpoint) / window |
|
| Deprecated_Endpoint | count(deprecated_endpoint) / window |
|
| Shadow_API | count(unknown_endpoint) / window |
|
| Endpoint_Latency | p95(endpoint_latency) |
|
| api_key_id | api_client_key |
|
| Key_Usage_Volume | count(api_key_use) / window |
|
| Invalid_Key_Rate | count(invalid_key) / count(api_key_use) |
|
| Key_Rotation | count(key_rotation) / window |
|
| Concurrent_Key_Use | uniq(concurrent_key_use) |
|
| response_size | bytes_out |
|
| Payload_Size_Avg | avg(response_size) |
|
| Data_Exfiltration | z(bytes_out) > threshold |
|
| Large_Payload | p99(response_size) |
|
| Zero_Byte_Response | count(response_size == 0) / window |
|
| rate_limit_status | count(429) / window |
|
| Throttled_Requests | count(429) / count(request) |
|
| Quota_Consumption | used_quota / allocated_quota |
|
| Abusive_Client | score(client_abuse) |
| Metric | Expression | Example |
|---|---|---|
| tls_version | enum(TLS1.0, TLS1.2, TLS1.3) |
|
| Legacy_Protocol | count(TLS1.0) / window |
|
| TLS_1_3_Adoption | count(TLS1.3) / count(tls_handshake) |
|
| Downgrade_Attack | count(downgrade_attempt) / window |
|
| cipher_suite | cipher_id |
|
| Weak_Cipher_Usage | count(weak_cipher) / window |
|
| Cipher_Distribution | distribution(cipher_suite) |
|
| PFS_Usage | count(pfs_cipher) / count(cipher_suite) |
|
| content_type | mime_type |
|
| MIME_Type_Mismatch | count(mime_mismatch) / window |
|
| Executable_Download | count(exec_download) / window |
|
| Unexpected_Content | count(unexpected_content) / window |
|
| encoding | charset_or_compression |
|
| Compression_Ratio | bytes_in / bytes_out |
|
| Double_Encoding | count(double_encode) / window |
|
| Malformed_Encoding | count(malformed_encode) / window |
| Metric | Expression | Example |
|---|---|---|
| session_id | session_key |
|
| Session_Count | count(session_id) / window |
|
| Session_Fixation | count(session_fixation) / window |
|
| Session_Churn | count(session_end) / window |
|
| Concurrent_Sessions | uniq(active_session_id) |
|
| session_duration | seconds_active |
|
| Avg_Session_Length | avg(session_duration) |
|
| Short_Sessions | count(session_duration < threshold) |
|
| Long_Sessions | count(session_duration > threshold) |
|
| Session_Timeout_Rate | count(timeout) / count(session_end) |
|
| login_status | enum(success, failure) |
|
| Login_Success_Rate | count(success) / count(login) |
|
| Brute_Force | z(failed_login_rate) |
|
| Credential_Stuffing | score(credential_stuffing) |
|
| Impossible_Travel | count(impossible_travel) / window |
|
| keepalive_status | enum(alive, dead) |
|
| Keepalive_Failures | count(keepalive_fail) / window |
|
| Zombie_Sessions | count(zombie_session) / window |
| Metric | Expression | Example |
|---|---|---|
| src_port | port_number |
|
| Ephemeral_Port_Exhaustion | z(src_port_distribution) |
|
| Fixed_Source_Port | count(src_port_fixed) / window |
|
| Port_Scan_Source | count(src_port_scan) / window |
|
| dest_port | service_port |
|
| Service_Distribution | distribution(dest_port) |
|
| Dark_Port_Access | count(dest_port_unexpected) / window |
|
| Port_Scan_Dest | count(dest_port_scan) / window |
|
| High_Port_Usage | count(dest_port_high) / window |
|
| tcp_flags | enum(SYN, ACK, FIN, RST) |
|
| SYN_Flood | rate(SYN) > baseline |
|
| RST_Rate | count(RST) / count(tcp_flags) |
|
| Null_Scan | count(null_scan) / window |
|
| Xmas_Scan | count(xmas_scan) / window |
|
| Handshake_Completion | count(handshake_ok) / count(handshake_start) |
|
| window_size | tcp_window |
|
| Zero_Window | count(window_size == 0) / window |
|
| Window_Scaling | avg(window_scale) |
|
| Retransmission_Correlation | corr(retransmissions, latency) |
|
| retransmission_rate | count(retransmit) / count(packet) |
|
| Retransmission_Spike | z(retransmission_rate) |
|
| High_Retransmission_Host | top(retransmission_host) |
|
| Global_Retransmission | avg(retransmission_rate) |
| Metric | Expression | Example |
|---|---|---|
| flow_bytes_in | bytes_in |
|
| Inbound_Volume | sum(flow_bytes_in) |
|
| Large_Transfer | p99(flow_bytes_in) |
|
| Volume_Spike | z(flow_bytes_in) |
|
| Ratio_Analysis | flow_bytes_out / flow_bytes_in |
|
| flow_bytes_out | bytes_out |
|
| Outbound_Volume | sum(flow_bytes_out) |
|
| Exfiltration_Detection | z(flow_bytes_out) |
|
| Upload_Anomaly | z(flow_bytes_out / flow_duration) |
|
| Asymmetric_Flow | z(flow_bytes_out / flow_bytes_in) |
|
| flow_packets | packet_count |
|
| Packet_Volume | sum(flow_packets) |
|
| Small_Packet_Flood | count(flow_packets < threshold) |
|
| Packet_Size_Avg | avg(flow_bytes_out / flow_packets) |
|
| Scan_Detection | score(scan_pattern) |
|
| flow_duration | seconds_active |
|
| Average_Flow_Duration | avg(flow_duration) |
|
| Long_Lived_Flows | count(flow_duration > threshold) |
|
| C2_Beaconing | score(beacon_pattern) |
|
| Tunnel_Detection | score(tunnel_pattern) |
|
| flow_start_time | timestamp_start |
|
| Flow_Start_Distribution | distribution(flow_start_time) |
|
| Off_Hours_Activity | count(off_hours_flow) / window |
|
| Burst_Detection | z(burst_rate) |
|
| Time_Correlation | corr(flow_start_time, flow_end_time) |
|
| flow_end_reason | enum(RST, FIN, TIMEOUT) |
|
| End_Reason_Distribution | distribution(flow_end_reason) |
|
| Timeout_Flows | count(flow_end_reason == TIMEOUT) |
|
| RST_FIN_Analysis | count(RST) / count(FIN) |
|
| Forced_Closure | count(forced_close) / window |
|
| application_id | dpi_app_id |
|
| App_Distribution | distribution(application_id) |
|
| Shadow_IT_Detection | count(unknown_app) / window |
|
| New_Application | count(new_app) / window |
|
| App_Usage_Trend | trend(app_usage) |
|
| flow_direction | enum(ingress, egress, internal) |
|
| Direction_Distribution | distribution(flow_direction) |
|
| Egress_Anomaly | z(egress_ratio) |
|
| Lateral_Movement | score(lateral_pattern) |
|
| Internal_Traffic | rate(internal_flow) |
| Metric | Expression | Example |
|---|---|---|
| src_ip | source_identity |
|
| Unique_Sources | uniq(src_ip) |
|
| New_Source_Detection | count(new_src_ip) / window |
|
| Top_Talkers | top(src_ip_by_volume) |
|
| Source_Reputation | score(src_ip_reputation) |
|
| Internal_vs_External | ratio(internal, external) |
|
| dest_ip | destination_target |
|
| Unique_Destinations | uniq(dest_ip) |
|
| New_Destination_Alert | count(new_dest_ip) / window |
|
| Destination_Reputation | score(dest_ip_reputation) |
|
| Beaconing_Detection | score(beaconing_pattern) |
|
| Rare_Destination_Access | z(dest_ip_rarity) |
|
| latency | rtt_ms |
|
| Average_Latency | avg(latency) |
|
| P50_Latency | p50(latency) |
|
| P95_Latency | p95(latency) |
|
| P99_Latency | p99(latency) |
|
| Latency_Anomaly | z(latency) |
|
| Latency_Trend | trend(latency) |
|
| packet_loss | loss_ratio |
|
| Loss_Rate | avg(packet_loss) |
|
| Loss_Spike | z(packet_loss) |
|
| Loss_Outliers | count(packet_loss > threshold) |
|
| Loss_by_Path | group(path, avg(packet_loss)) |
|
| P99_Loss | p99(packet_loss) |
|
| hop_count | ttl_hops |
|
| Avg_Path_Length | avg(hop_count) |
|
| Path_Change_Detection | count(path_change) / window |
|
| Excessive_Hops | count(hop_count > threshold) |
|
| TTL_Expiry_Rate | count(ttl_expired) / window |
|
| tos_dscp | qos_tag |
|
| QoS_Marking_Distribution | distribution(tos_dscp) |
|
| Voice_Traffic_Tagging | count(voice_tag) / window |
|
| Mismarked_Traffic | count(mismark) / window |
|
| protocol_id | enum(TCP, UDP, ICMP, GRE, ESP) |
|
| Protocol_Distribution | distribution(protocol_id) |
|
| Unusual_Protocol | z(protocol_id_rarity) |
|
| ICMP_Volume | rate(ICMP) |
|
| GRE_ESP_Tunnels | count(GRE_or_ESP) / window |
|
| icmp_type | enum(unreachable, echo_request) |
|
| Unreachable_Rate | count(icmp_unreachable) / count(icmp_type) |
|
| Echo_Request_Volume | rate(icmp_echo) |
|
| ICMP_Flood_Detection | z(icmp_echo_rate) |
|
| Redirect_Messages | count(icmp_redirect) / window |
|
| bgp_peer_state | enum(established, idle) |
|
| Peer_Status | count(bgp_peer_state) / window |
|
| State_Flap_Detection | count(bgp_flap) / window |
|
| Idle_Peer_Alert | count(bgp_idle) / window |
|
| Prefix_Count_Change | z(prefix_count) |
|
| Session_Uptime | avg(bgp_session_uptime) |
|
| route_next_hop | next_hop_ip |
|
| Next_Hop_Distribution | distribution(route_next_hop) |
|
| Next_Hop_Change | count(next_hop_change) / window |
|
| Black_Hole_Routes | count(blackhole_route) / window |
|
| Path_Symmetry | score(path_symmetry) |
|
| tunnel_id | sdwan_tunnel_id |
|
| Tunnel_Status | count(tunnel_up) / window |
|
| Tunnel_Flap_Detection | count(tunnel_flap) / window |
|
| Tunnel_Latency | avg(tunnel_latency) |
|
| Tunnel_Throughput | avg(tunnel_throughput) |
|
| Failover_Events | count(failover) / window |
|
| vpc_id | cloud_vpc_id |
|
| VPC_Traffic_Distribution | distribution(vpc_id) |
|
| Cross_VPC_Traffic | count(cross_vpc) / window |
|
| New_VPC_Detection | count(new_vpc) / window |
|
| VPC_Flow_Anomaly | z(vpc_flow_rate) |
|
| security_group_id | cloud_sg_id |
|
| SG_Rule_Effectiveness | score(sg_effectiveness) |
|
| Overly_Permissive_SG | count(overly_permissive_sg) / window |
|
| SG_Change_Detection | count(sg_change) / window |
|
| Unused_SG_Detection | count(unused_sg) / window |
|
| SG_Deny_Spike | z(sg_deny_rate) |
| Metric | Expression | Example |
|---|---|---|
| src_mac | mac_address |
|
| Unique_MACs | uniq(src_mac) |
|
| New_MAC_Detection | count(new_mac) / window |
|
| MAC_Spoofing_Detection | count(mac_spoof) / window |
|
| OUI_Distribution | distribution(oui) |
|
| Rogue_Device_Detection | count(rogue_device) / window |
|
| vlan_id | vlan_identifier |
|
| VLAN_Distribution | distribution(vlan_id) |
|
| VLAN_Hopping_Detection | count(vlan_hop) / window |
|
| Native_VLAN_Traffic | rate(native_vlan) |
|
| Unused_VLAN_Detection | count(unused_vlan) / window |
|
| interface | port_identifier |
|
| Port_Utilization | avg(port_util) |
|
| Port_Flapping | count(port_flap) / window |
|
| Broadcast Storm | z(broadcast_rate) |
|
| Port_Error_Rate | count(port_error) / window |
|
| Duplex_Mismatch | count(duplex_mismatch) / window |
|
| frame_type | enum(Ethernet_II, 802.3) |
|
| Frame_Type_Distribution | distribution(frame_type) |
|
| Unusual_EtherType | count(unknown_ethertype) / window |
|
| ARP_Traffic_Volume | rate(arp) |
|
| IPv6_Adoption | count(ipv6) / count(frame_type) |
|
| stp_state | enum(blocking, forwarding) |
|
| Blocking_Port_Count | count(stp_blocking) / window |
|
| STP_Topology_Change | count(stp_change) / window |
|
| Root_Bridge_Change | count(root_bridge_change) / window |
|
| Port_State_Flap | count(stp_flap) / window |
|
| Designated_Port_Ratio | count(designated_port) / count(port) |
|
| link_status | enum(up, down) |
|
| Link_Availability | count(up) / window |
|
| Link_Down_Events | count(down) / window |
|
| Flapping_Detection | count(link_flap) / window |
|
| Critical_Link_Monitor | count(critical_link_down) / window |
|
| MTTR | mean(time_to_recover) |
|
| input_discards | count(input_discard) / window |
|
| Discard_Rate | input_discards / window |
|
| Discard_Spike | z(input_discards) |
|
| input_errors | count(input_error) / window |
|
| Error_Rate | input_errors / window |
|
| CRC_Error_Spike | z(crc_error) |
|
| Error_Trend | trend(input_errors) |
|
| Hardware_Failure | count(hardware_failure) / window |
|
| Error_Distribution | distribution(error_type) |
|
| neighbor_mac | lldp_cdp_neighbor |
|
| Expected_Neighbors | count(expected_neighbor) / window |
|
| Neighbor_Change | count(neighbor_change) / window |
|
| Missing_Neighbor | count(missing_neighbor) / window |
|
| New_Neighbor_Detection | count(new_neighbor) / window |
|
| arp_status | enum(resolved, incomplete) |
|
| Incomplete_ARP_Rate | count(incomplete_arp) / count(arp) |
|
| ARP_Timeout_Spike | z(arp_timeout) |
|
| ARP_Cache_Size | avg(arp_cache_size) |
|
| Duplicate_IP_Detection | count(duplicate_ip) / window |
| Metric | Expression | Example |
|---|---|---|
| rssi | signal_strength |
|
| Average_RSSI | avg(rssi) |
|
| P10_RSSI | p10(rssi) |
|
| Low_Signal_Clients | count(rssi < threshold) |
|
| RSSI_Anomaly | z(rssi) |
|
| Coverage_Holes | count(coverage_gap) / window |
|
| RSSI_Distribution | distribution(rssi) |
|
| snr | signal_to_noise |
|
| Average_SNR | avg(snr) |
|
| SNR_Anomaly | z(snr) |
|
| Low_SNR_Clients | count(snr < threshold) |
|
| SNR_vs_Throughput | corr(snr, throughput) |
|
| channel | channel_id |
|
| Channel_Utilization | avg(channel_util) |
|
| Co_Channel_Interference | z(co_channel_interference) |
|
| Channel_Change_Rate | count(channel_change) / window |
|
| DFS_Event_Rate | count(dfs_event) / window |
|
| Channel_Width | avg(channel_width) |
|
| data_rate | negotiated_rate |
|
| Average_Data_Rate | avg(data_rate) |
|
| P10_Data_Rate | p10(data_rate) |
|
| Low_Rate_Clients | count(data_rate < threshold) |
|
| Rate_vs_RSSI | corr(data_rate, rssi) |
|
| retry_rate | retries / frames |
|
| Average_Retry_Rate | avg(retry_rate) |
|
| Retry_Spike | z(retry_rate) |
|
| High_Retry_APs | top(retry_rate) |
|
| Retry_vs_Channel_Util | corr(retry_rate, channel_util) |
|
| noise_floor | rf_noise |
|
| Average_Noise_Floor | avg(noise_floor) |
|
| Interference_Spike | z(noise_floor) |
|
| High_Noise_APs | top(noise_floor) |
|
| Noise_Trend | trend(noise_floor) |
|
| optical_rx_power | rx_light_level |
|
| Rx_Power_Level | avg(optical_rx_power) |
|
| Low_Power_Alert | count(optical_rx_power < threshold) |
|
| Power_Degradation | trend(optical_rx_power) |
|
| Link_Margin | target_rx - optical_rx_power |
|
| Asymmetric_Power | abs(tx_power - rx_power) |
|
| transceiver_temp | temp_c |
|
| Average_Temperature | avg(transceiver_temp) |
|
| Overheating_Alert | count(transceiver_temp > threshold) |
|
| Temperature_Trend | trend(transceiver_temp) |
|
| Thermal_Runaway | z(transceiver_temp) |
|
| poe_power_draw | watts |
|
| Power_Per_Device | avg(poe_power_draw) |
|
| Total_Budget_Usage | sum(poe_power_draw) |
|
| Power_Anomaly | z(poe_power_draw) |
|
| Class_Mismatch | count(power_class_mismatch) / window |
|
| Power_Trend | trend(poe_power_draw) |
|
| fan_status | enum(ok, fail) |
|
| Fan_Health | count(ok) / window |
|
| Fan_Failure_Alert | count(fail) / window |
|
| Fan_Speed_Anomaly | z(fan_speed) |
|
| Degraded_Cooling | count(degraded_cooling) / window |
|
| psu_status | enum(ok, fail) |
|
| PSU_Health | count(ok) / window |
|
| PSU_Failure | count(fail) / window |
|
| Redundancy_Status | score(redundancy) |
|
| Power_Input_Voltage | avg(input_voltage) |
|
| Load_Balance | score(load_balance) |
| Metric | Expression | Example |
|---|---|---|
| process_name | executable_name |
|
| Process_Execution_Volume | count(process_start) / window |
|
| Rare_Process_Detection | z(process_rarity) |
|
| Process_Spawn_Rate | rate(process_spawn) |
|
| Living_Off_the_Land | score(lotl_usage) |
|
| process_hash | sha256_or_md5 |
|
| Known_Malware_Match | count(known_hash) / window |
|
| Unknown_Hash_Detection | count(unknown_hash) / window |
|
| Hash_Diversity | uniq(process_hash) |
|
| First_Seen_Hash | count(new_hash) / window |
|
| parent_process | parent_exec |
|
| Process_Tree_Anomaly | score(process_tree_anomaly) |
|
| Suspicious_Spawning | count(suspicious_spawn) / window |
|
| Injection_Detection | count(injection) / window |
|
| Execution_Chain_Length | avg(exec_chain_length) |
|
| process_cmd_line | full_command |
|
| Encoded_Command | count(encoded_cmd) / window |
|
| Long_Command_Line | z(cmd_length) |
|
| Suspicious_Patterns | score(cmd_pattern) |
|
| PowerShell_Cmdlets | count(ps_cmdlet) / window |
|
| registry_key | reg_path |
|
| Run_Key_Modifications | count(run_key_change) / window |
|
| Service_Registry_Changes | count(service_reg_change) / window |
|
| Persistence_Detection | score(persistence_indicators) |
|
| Unusual_Key_Access | count(unusual_reg_access) / window |
|
| file_operation | enum(create, modify, delete) |
|
| File_Operations_Volume | count(file_op) / window |
|
| Mass_File_Changes | z(file_op_rate) |
|
| Sensitive_File_Access | count(sensitive_file) / window |
|
| Shadow_Copy_Deletion | count(shadow_copy_delete) / window |
|
| network_connection_local | local_conn |
|
| Outbound_Connections | count(outbound_conn) / window |
|
| Rare_Destination | z(dest_rarity) |
|
| Beaconing_Detection | score(beaconing_pattern) |
|
| Port_Anomaly | z(port_rarity) |
|
| usb_device_id | usb_key |
|
| USB_Insert_Volume | count(usb_insert) / window |
|
| Unknown_USB_Device | count(unknown_usb) / window |
|
| USB_Write_Volume | count(usb_write) / window |
|
| After_Hours_USB | count(usb_after_hours) / window |
Layer 2: Meso Analysis¶
Physics Analogy
Statistical mechanics—the emergence of macroscopic properties from microscopic chaos. Just as temperature emerges from the average kinetic energy of particles, ARQ dimensions emerge from the statistical properties of telemetry events.
Aggregation Functions¶
The aggregation engine reduces cardinality while preserving signal. This is where millions become meaningful:
| Function | Purpose | Example | Preserves |
|---|---|---|---|
SUM |
Total volume | Total errors | Magnitude |
AVG |
Central tendency | Mean latency | Typical behavior |
MEDIAN |
Robust center | Median response time | Typical behavior |
MIN |
Lower bound | Min response time | Best-case behavior |
MAX |
Upper bound | Max response time | Worst-case behavior |
MODE |
Most common value | Most frequent status code | Typical behavior |
PERCENTILE(95) |
Tail behavior | P95 response time | Worst cases |
PERCENTILE(99) |
Extreme cases | P99 latency | Outliers |
COUNT_DISTINCT |
Cardinality | Affected users | Scope |
STDDEV |
Variability | Latency consistency | Stability |
HISTOGRAM(bucket) |
Distribution shape | Latency histogram | Spread & density |
ROLLUP |
Hierarchical aggregation | By service → region → org | Scope alignment |
TOPK(k) |
Highest contributors | Top 10 noisy hosts | Concentration |
SMA(window) |
Simple moving average | SMA(5m) latency | Trend smoothing |
EMA(alpha) |
Weighted moving average | EMA(0.3) error rate | Trend smoothing |
RATE |
Velocity | Requests/second | Throughput |
DELTA |
Change over time | Error rate delta | Change detection |
Statistical Normalization¶
To compare "apples to oranges" (latency in ms vs. error rates in %), we apply Z-score normalization:
Where:
- \(x\) = Raw observed value
- \(\mu\) = Historical mean (rolling 7-day)
- \(\sigma\) = Historical standard deviation
Interpretation
| Z-Score | Interpretation | Action |
|---|---|---|
| -2 to +2 | Normal variance | Continue monitoring |
| +2 to +3 | Notable deviation | Investigate |
| > +3 or < -3 | Significant anomaly | Alert |
| > +4 or < -4 | Critical anomaly | Auto-remediate |
ARQ Dimension Calculation¶
Raw technical capability score composed of Accessibility (40%), Retainability (30%), and Quality (30%).
Weight: 40% - Measures the ability to establish initial connection.
| Signal | Why it matters | Contribution |
|---|---|---|
| Physical layer connectivity and signal strength | Confirms the link is viable before higher layers can succeed. | Stabilizes first-contact reliability and reduces retries. |
| Network address allocation success rate | Ensures devices can obtain a usable network identity. | Prevents early session failures and onboarding drop-offs. |
| Identity verification and access control | Validates the requester and policy compliance at the edge. | Filters invalid access while keeping legitimate access fast. |
| Name resolution service availability | Guarantees services can be discovered by clients. | Removes the most common early-failure point in sessions. |
| Initial data transmission success rate | Confirms the first payload crosses the link cleanly. | Sets the baseline for downstream session continuity. |
Weight: 30% - Measures the ability to maintain connection.
| Signal | Why it matters | Contribution |
|---|---|---|
| Connection stability over time | Long-lived sessions are sensitive to jitter and drops. | Preserves continuity for real user workflows. |
| Seamless transition between access points | Mobility without interruption prevents session resets. | Sustains engagement during roaming or handoffs. |
| Overall connection quality metrics | Captures sustained performance beyond initial access. | Keeps sessions usable under real load conditions. |
| Automatic recovery from failures | Fast recovery reduces user-visible interruptions. | Converts transient faults into acceptable blips. |
Weight: 30% - Measures the quality of the connection.
| Signal | Why it matters | Contribution |
|---|---|---|
| Round-trip time and response speed | Latency dominates perceived responsiveness. | Keeps interactions crisp and predictable. |
| Data transfer rate and bandwidth | Throughput governs task completion time. | Sustains heavy workflows without bottlenecks. |
| End-to-end application responsiveness | Measures real service behavior, not just transport. | Aligns technical performance with user outcomes. |
| Perceived quality from user perspective | Captures the human judgment of the experience. | Anchors Q to actual satisfaction signals. |
Risk Deflation¶
Risk acts as friction—it opposes the positive effects of good performance:
| Risk Factor | Detection Source | Severity Multiplier |
|---|---|---|
| Active CVEs (Critical) | Vuln scanner | 0.3 |
| Active CVEs (High) | Vuln scanner | 0.15 |
| Anomalous Traffic | ML detector | 0.2 |
| Compliance Violation | Policy engine | 0.25 |
| Certificate Issues | TLS monitor | 0.1 |
| Data Exposure | DLP | 0.5 |
Layer 3: Macro Intelligence¶
Physics Analogy
Celestial mechanics and gravity—the E-score represents the gravitational pull of a digital experience, attracting or repelling users based on its strength.
Global Context¶
The Experience Score exists within a broader context that modulates its interpretation:
flowchart TB
subgraph CONTEXT["Global Context Factors"]
MKT[📈 Market Sentiment<br/><small>Social, news, analyst</small>]
REG[🌍 Regional Events<br/><small>Sports, politics, weather</small>]
CMP[🏢 Competitive Status<br/><small>Outages, launches, pricing</small>]
SEA[📅 Seasonality<br/><small>Holidays, cycles, patterns</small>]
end
subgraph ADJUST["Context Multiplier"]
CALC[Calculate<br/>context_mult]
end
MKT --> CALC
REG --> CALC
CMP --> CALC
SEA --> CALC
CALC --> |0.8 - 1.2| ESCORE[Experience Score]
classDef dark fill:#121922,stroke:#cfd8e3,color:#f5f5f5;
class MKT,REG,CMP,SEA,CALC,ESCORE dark;
style CONTEXT fill:#121922,stroke:#cfd8e3,color:#f5f5f5
style ADJUST fill:#121922,stroke:#cfd8e3,color:#f5f5f5
| Context Factor | Data Source | Range | Example Impact |
|---|---|---|---|
| Market Sentiment | News API, social | ±10% | Negative press → stricter threshold |
| Regional Events | Calendar, traffic | ±15% | Major event → higher expected load |
| Competitor Status | Monitoring, news | ±5% | Competitor outage → relative advantage |
| Seasonality | Historical patterns | ±20% | Holiday spike → adjusted baseline |
Experience Score Formula¶
The complete formula integrating all layers:
Where:
| Variable | Description | Range |
|---|---|---|
| \(A, R, Q\) | ARQ dimension values | 0.0 - 1.0 |
| \(w_A, w_R, w_Q\) | Dynamic weights | Sum to 1.0 |
| \(D_{risk}\) | Risk deflation factor | 0.0 - 1.0 |
| \(C_{context}\) | Context multiplier | 0.8 - 1.2 |
| \(E\) | Experience Score | 0 - 100 |
Full Calculation Example
# Input telemetry (aggregated)
Accessibility metrics:
- Uptime: 99.95%
- DNS Success: 99.99%
- Connection Rate: 99.8%
A = (0.9995 × 0.9999 × 0.998)^(1/3) = 0.9991
Retainability metrics:
- Session Duration: 8.5 min (target: 10 min) → 0.85
- Completion Rate: 94%
- Recovery Rate: 88%
R = (0.85 × 0.94 × 0.88)^(1/3) = 0.8893
Quality metrics:
- P95 Response: 180ms (target: 200ms) → 0.90
- Throughput: 95% of capacity
- Render: 92% within threshold
Q = (0.90 × 0.95 × 0.92)^(1/3) = 0.9231
# Weights (enterprise segment)
w_A = 0.30, w_R = 0.40, w_Q = 0.30
# Risk assessment
- 2 medium CVEs: 0.15 × 2 = 0.30
- Minor compliance gap: 0.05
Risk Score = 35
D_risk = 1 - 0.35 = 0.65
# Context
- Normal market conditions
- No regional events
- Competitor stable
C_context = 1.0
# Final calculation
ARQ_weighted = (0.9991 × 0.30) + (0.8893 × 0.40) + (0.9231 × 0.30)
ARQ_weighted = 0.2997 + 0.3557 + 0.2769 = 0.9323
E = 0.9323 × 0.65 × 1.0 × 100 = 60.6
Integration Points¶
-
Visualize how telemetry flows through the three-layer architecture in real-time.
-
Understand the mathematical space where ARQ dimensions create the trust manifold.
-
The formal specification for telemetry collection and sensor configuration.
-
JSON schema for configuring telemetry collection agents.