Skip to content

MODULE 1 · LESSON 5 · Data foundations

Structured versus unstructured data

Explain how turning text into a table creates assumptions that need checking.

By Laurence Svekis · Self-paced · All worked data is synthetic unless you supply your own source.

Try the practice ↓

PredictInspectCalculateCheckExplainTransfer

Your starting point

Prerequisites: Use the ideas and vocabulary from lesson 4; revisit its worked examples if needed.

Definition: Structured data follows an explicit arrangement of fields and records. Unstructured data, such as free-form notes, does not arrive in that consistent tabular form. Semi-structured formats have labels or hierarchy but may not share a uniform set of fields.

Structure is a design choice

A table of ticket IDs, channels and durations is structured. A paragraph describing a customer’s experience contains information, but its meaning is not reduced to a fixed set of fields. A JSON document can have named properties and nested lists, making it semi-structured for many analytic workflows. The distinction concerns organization, not whether the data is useful or trustworthy.

Extraction changes the evidence

Converting comments into categories requires a coding rule. If a comment says “delivery was late but support was excellent,” forcing it into one positive-or-negative label discards a meaningful distinction. A better design may allow separate delivery and support fields, multiple labels or a mixed category. Keep the original text when permission and privacy allow, so the coding can be reviewed.

Treat AI extraction as a proposal

An AI assistant may classify text consistently in appearance while missing sarcasm, context or a changed definition. Test a small labeled sample, compare disagreements and revise the coding guide. Do not assume a model’s confidence wording is a calibrated probability. Record ambiguous cases and an unknown category rather than forcing every record into a convenient label. Automation is useful when its transformations remain inspectable.

Two worked examples

From notes to categories

Three fictional notes say “late arrival,” “arrived on time” and “late arrival, helpful driver.” A delay flag can be yes, no, yes, while a service-comment field preserves the praise. A single sentiment label would combine two different topics. Write the extraction rule before counting how often lateness is mentioned.

Nested orders

An order record contains one customer and a list of three purchased items. Flattening it to three item rows repeats the order-level delivery fee. Summing that fee after flattening would count it three times. Preserve the distinction between order-level and item-level fields before building a conventional table.

Where this becomes useful

A workshop organizer wants to summarize open comments. A small coding guide with examples and ambiguity rules is more defensible than a word cloud that treats every occurrence as equivalent evidence of satisfaction.

Make an attempt

Use the Data Type Classifier, then create three fictional support notes in your lesson workbook. Propose a two-column extraction scheme, label the notes manually, and identify a sentence that the scheme cannot represent well.

Open the accompanying interactive lab →

Misconception check

Claim: Unstructured means unusable.

Inspect the correction

Text, images and audio can be useful evidence, but extraction and interpretation require explicit methods and validation.

Independent application

Write five fictional comments about a service. Build a coding guide with definitions, two examples and an ambiguity rule. Have another person code them independently or revisit them later, then document disagreements before reporting counts.

Completion records practice and self-review. It is not certification of statistical competence.