By Paul R Salmon FCILT, FSCM
In the world of supply chain management, we often take comfort in the assumption that a stock-keeping unit (SKU) or, in the defence context, a NATO Stock Number (NSN), is a unique identifier. It’s a neat, 13-digit string that gives logisticians the confidence that one number equals one item.
But what if that assumption is wrong?
Duplication in SKU data is far more common than many supply chain professionals realise. Across industries – from e-commerce giants like Amazon to defence organisations managing millions of items – SKU duplication creates hidden costs, bloated inventories, and operational inefficiencies.
The good news? Artificial Intelligence (AI) is changing the game. By applying AI-driven tools to identify and resolve duplication, organisations can streamline operations, reduce costs, and unlock new levels of supply chain resilience.
Let’s explore how and why.
📦 The Illusion of Uniqueness in Supply Chains
At first glance, a SKU or NSN seems unambiguous – a single number for a single item. But in practice, duplication often creeps in:
Variant Proliferation: Small differences, such as colour or packaging, result in multiple SKUs for effectively the same product. Supplier Diversity: Two vendors providing functionally identical items are often assigned separate identifiers. Codification Errors: Human error, inconsistent data entry, and legacy system migrations lead to duplicate records. Parallel Procurement: Different business units or countries procure the same item under different identifiers.
In defence logistics, this can mean two NSNs for identical bolts, batteries, or filters. In retail, it might mean dozens of SKUs for the same T-shirt, just because descriptions vary slightly across suppliers.
The result? Redundant inventory, procurement inefficiencies, and a distorted view of true demand.
📚 Civilian Case Study 1: Amazon and SKU Duplication
Amazon’s marketplace handles billions of SKUs, many submitted directly by sellers. Slight variations in product descriptions often create multiple listings for the same product.
Example: A USB-C cable might appear under SKUs such as:
USB-C Charging Cable 1m Black 1 Meter Black USB-C Cable Type-C Charger Cord Black 1M
While identical in function, these duplicates confuse customers, fragment reviews, and distort Amazon’s inventory planning. Amazon has invested heavily in AI tools to de-duplicate SKUs, collapsing redundant listings and improving customer experience.
🛞 Civilian Case Study 2: Automotive Parts Commonality
In the automotive industry, SKU duplication drives complexity. A major car manufacturer discovered it had three SKUs for identical spark plugs used across different vehicle models. These were procured separately, stocked in separate warehouses, and maintained as if they were unique.
By applying AI and machine learning to parts data, the company identified over 20% duplication across its spare parts catalogue. Consolidation saved millions annually by reducing inventory holding costs and streamlining supplier relationships.
🧑💻 Why Traditional Methods Fail
Spotting duplication in millions of records is no easy feat. Traditional methods rely on exact matches in part numbers or descriptions, which rarely work in practice. Human codifiers and analysts may manually check for duplicates, but this process is:
❌ Time-Consuming – reviewing vast datasets is not scalable.
❌ Error-Prone – humans struggle with inconsistent data or variations in spelling, units, and languages.
❌ Static – rules-based systems can’t adapt to new patterns of duplication.
Enter AI.
🤖 How AI Finds Duplicates Humans Miss
AI and machine learning bring a new toolkit to the duplication challenge:
✅ Natural Language Processing (NLP)
AI can read item descriptions, technical specifications, and supplier catalogues, even when written differently, and identify potential matches. For instance, “Screw, Hex Head, 5mm” and “5MM Hexagonal Bolt” may seem different to a rules-based system but identical to an AI model.
✅ Fuzzy Matching
AI doesn’t need exact matches. It recognises approximate similarities, accounting for typos, synonyms, or unit conversions (e.g., “100mm” vs. “10cm”).
✅ Image Recognition
Where photos or diagrams exist, computer vision can spot visual similarities, flagging items that look identical but are catalogued separately.
✅ Cross-Dataset Analysis
AI can compare usage data, supplier information, and demand patterns to infer duplication invisible in siloed systems.
✅ Continuous Learning
As supply chain teams validate AI-suggested duplicates, the model improves over time, reducing false positives and becoming smarter.
🛠 Defence Supply Chains: A Hidden Problem
In defence, the stakes are even higher.
With millions of NSNs in the NATO Codification System, duplication is inevitable. For example:
Two NSNs assigned to functionally identical batteries because of different suppliers. Slight differences in packaging or connector type triggering new stock numbers. Obsolete items recoded instead of rationalised, creating legacy duplication.
This leads to:
Excess inventory holding, as multiple SKUs are stocked unnecessarily. Reduced platform availability, when stocks are spread across non-interchangeable variants. Procurement inefficiencies, with fragmented vendor relationships.
By deploying AI tools, defence organisations can rationalise stock holdings, eliminate redundant NSNs, and focus on true critical spares.
💡 The Payoff
AI-driven de-duplication delivers tangible benefits:
Reduced Inventory Costs: By consolidating SKUs, organisations carry fewer redundant items. Improved Forecast Accuracy: Demand signals are clearer when duplication is eliminated. Streamlined Procurement: Larger consolidated orders drive better pricing and supplier terms. Enhanced Availability: Focus on fewer variants improves supply resilience.
In a contested logistics environment, where agility and efficiency are vital, these advantages can be mission-critical.
🧠 Think Again: Is Your SKU Really Unique?
The humble SKU or NSN may no longer be the safe anchor point we thought it was. In complex, global supply chains, duplication is a hidden but costly problem.
AI gives us the power to challenge assumptions, clean up duplication, and build smarter, leaner, and more resilient supply chains – whether in retail, automotive, or defence.
The question for supply chain leaders isn’t if duplication exists in their catalogues. It’s how much, and what are you going to do about it?
Leave a Reply