Navy Automated Data Cleansing with ML

Customer Challenge

Poor data quality is hindering the Department of Navy’s (DON) ability to gain valuable and accurate insight from their data. Given the volume of errors, manual correction is ineffective and inefficient. 

Innovative Solution

ILW data scientists implemented Phase I of our Automated Data Cleansing and Analysis Tool (ADCAT), which applies machine learning (ML) and probabilistic graphical modeling (PGM) to automatically cleanse DON data of errors.

For Phase II, ILW is currently applying algorithm enhancements and user interface creation for improved healing functionality across multiple commands, domains, and DON operational environments. 

Benefits/Outcomes

  • Robust natural language processing (NLP) and supervised ML classifier algorithm resulting in 71-87% error correction rates
  • PGM Bayesian network in ADCAR to provide end-users with the five most probable corrections for a given error. 95% of the time, the correct value for an error was in the top five most probable values
  • A human-in-the-loop error correction recommendation solution is available when needed to enable review and validation of the predictions

Business Value

  • Improved analyst productivity: less time correcting data, increased focus on core mission tasks
  • Higher quality data: higher-confidence, data-informed decisions, cost savings

Toolbox

  • Supervised/unsupervised ML with over 16,000 parameter combinations tested
  • Probabilistic graphical model (Bayesian)
  • Open-source Python solution using DoD compatible libraries
  • Categorical, ordinal, and string data types
  • NAVAIR maintenance data (S3 aircraft)
  • NAVSEA labor data

Related Case Studies You May Like

Data Engineering & Data Science

Data Engineering & Data Science

Paint Hangar IoT Monitoring

Paint Hangar IoT Monitoring

Pre-Flight Inspection AR Application

Pre-Flight Inspection AR Application

Expert Capture Maintenance Training Pilot

Expert Capture Maintenance Training Pilot

Navy Automated Data Cleansing with ML

Navy Automated Data Cleansing with ML

Automated Data Capture and Prediction

Automated Data Capture and Prediction

Automated Data Crosswalks

Automated Data Crosswalks

Data Framework for Air Force Logistics

Data Framework for Air Force Logistics

Contract Conversion & Analytics

Contract Conversion & Analytics

Database Tuning & Optimization

Database Tuning & Optimization

Augmented Reality Tools to Increase Workforce Productivity Across the Enterprise

Augmented Reality Tools to Increase Workforce Productivity Across the Enterprise

Decision Support for Cyber Hygiene

Decision Support for Cyber Hygiene

Expert Capture Maintenance Training Pilot

Expert Capture Maintenance Training Pilot

Augmented Reality Engineering Collaboration

Augmented Reality Engineering Collaboration

Big Data Ingestion & Cloud Architecture

Big Data Ingestion & Cloud Architecture

Cloud-Native Azure PaaS Architecture

Cloud-Native Azure PaaS Architecture

Azure Data Integration Hub Modernization

Azure Data Integration Hub Modernization

Data Warehousing & Business Intelligence

Data Warehousing & Business Intelligence

App Service Azure Infrastructure

App Service Azure Infrastructure

Big Data Engineering for Improved Analytics

Big Data Engineering for Improved Analytics

Agile Big Data Development

Agile Big Data Development

Cloud-Based Big Data Analytics

Cloud-Based Big Data Analytics

Supply Chain Predictive Analytics

Supply Chain Predictive Analytics

Cost Allocation Rules Engine Modernization

Cost Allocation Rules Engine Modernization

Data Services Cloud Migration Support

Data Services Cloud Migration Support

Predictive Analytics for the Aircraft Digital Thread

Predictive Analytics for the Aircraft Digital Thread

Automated Data Capture and Prediction

Automated Data Capture and Prediction

On-Demand Maintenance Analytics

On-Demand Maintenance Analytics

Algorithm Development & Text Analytics

Algorithm Development & Text Analytics

Machine Learning & NLP for Decision Support

Machine Learning & NLP for Decision Support

Sensor Data Analysis for Predictive CBM+

Sensor Data Analysis for Predictive CBM+

Cutting-Edge Responsive Design

Cutting-Edge Responsive Design

Data Cleansing and Migration

Data Cleansing and Migration

Application Modernization

Application Modernization

Large-Scale Data Integration

Large-Scale Data Integration

Data Science Big Data Ingestion

Data Science Big Data Ingestion

Modern Analytics Framework

Modern Analytics Framework

Agile Big Data Analytics Framework

Agile Big Data Analytics Framework

Big Data Hadoop Administration

Big Data Hadoop Administration

Modern Data Ingestion Framework

Modern Data Ingestion Framework

Performance Tuning & Best Practices

Performance Tuning & Best Practices

Engines Forecast Reporting Tool

Engines Forecast Reporting Tool

Augmented Reality Combustion Chamber & Gear Pump Disassembly

Optimization Using Hadoop

Data Quality & Lineage Mapping

Big Data Platform Analytics Outcomes

Modern Analytic Framework

Data Cleansing and Migration

Enterprise Data Warehousing

Value-Driven Analytics

Enterprise Data Exchange

Valuable Insight into Customer Shopping Behaviors

Interested In Working With Us?