NLP: Real Candidate-Reported Questions

Source: ZS Associates — Data Science Associate

I would first understand the task and then build a task-specific preprocessing pipeline.

Typical steps are:

1. Remove or normalize unwanted HTML, URLs and special characters.
2. Handle punctuation appropriately.
3. Handle numbers based on whether they contain useful information.
4. Normalize case if appropriate.
5. Tokenize the text.
6. Handle spelling variations, contractions and abbreviations if required.
7. Apply stemming or lemmatization when appropriate.
8. Remove stopwords only when they are not useful for the task.
9. Convert the processed text into numerical features such as TF-IDF, n-grams or embeddings.

The important point is that I would not blindly remove every number, punctuation mark or stopword because those can carry useful information.

Source: ZS Associates — Data Science Associate

Yes.

Numbers can contain useful information depending on the problem.

For example:

1 star due to functional performance digital watch
2 star due to asthetic look
Iphone 17 pro worth 5 star
Iphone 17 worth 4 star
Apple watch 2.0 5 star
2026 internship
5 years experience
20% discount
iPhone 15 3 star
COVID-19

Removing all numbers could remove useful predictive signals.

Instead, I would determine whether numbers have semantic value and either:

• preserve them,
• normalize them,
• replace them with special tokens, or
• remove them only when they are irrelevant.

Source: ZS Associates — Data Science Associate

Yes.

Instead of applying something like:

re.sub('[^a-zA-Z]', ' ', text)

blindly, I would decide what should be retained based on the task.

For example:

COVID-19
20%
₹999
C++
Python 3.12

can contain useful information.

A better approach is to selectively normalize unwanted characters while preserving meaningful tokens.

Source: ZS Associates — Data Science Associate

Stemming is generally faster and simpler because it uses heuristic rules to reduce words to a root-like form.

Lemmatization attempts to return a valid dictionary/base form and generally uses more linguistic information.

Example:

Studies → studi       # stemming example
Studies → study       # lemmatization

I would choose based on the task, dataset size, accuracy requirements and computational constraints.

Source: ZS Associates — Data Science Associate

Lemmatization is useful when maintaining a linguistically meaningful base word is important.

For example:

better → good
studies → study
running → run

Stemming may produce crude forms that are not valid words.

For simple large-scale text classification where speed is important, stemming can still be useful.

Source: American Express — NLP Data Scientist

I would not automatically remove everything.

I would first determine whether punctuation or numbers contain information for the task.

For example, in sentiment analysis:

Great!!!

the punctuation may contain useful sentiment information.

Similarly:

₹999
5-star
20%

may be important in a product-review problem.

The preprocessing rule should therefore be task-specific.

Source: American Express — NLP Data Scientist

I would first inspect the source text and encoding.

A practical approach is:

1. Ensure the input has the expected Unicode encoding.
2. Normalize Unicode when required.
3. Handle malformed characters.
4. Apply transliteration only when appropriate.
5. Validate the output after normalization.

I would avoid silently converting every character because that can destroy information in multilingual text.

Source: ZS Associates — Data Science Associate

Common approaches include:

Traditional representations

• One-hot encoding
• Bag of Words
• TF-IDF
• N-grams

Dense representations

• Word2Vec
• GloVe
• FastText
• Sentence embeddings
• Transformer embeddings

The choice depends on the problem, dataset size, semantic requirements and model architecture.

Source: ZS Associates — Data Science Associate

These are useful baseline representations.

Bag of Words

Represents a document using word frequencies.

TF-IDF

Weights terms according to their importance within a document relative to the corpus.

N-grams

Capture sequences of words.

For example:

"I love NLP"

Bigrams:

I love
love NLP

I would compare them experimentally using the same train/validation setup and evaluation metrics rather than assuming one will always perform best.

Source: Zee Entertainment — Data Scientist

A Term-Document Matrix represents the relationship between terms and documents.

Typically:

• Rows represent terms.
• Columns represent documents.
• Values represent frequency or another weighting such as TF-IDF.

Example:

             Doc1   Doc2
NLP           2      1
Python        1      3
AI            0      2

It provides a numerical representation of text that can be used by traditional ML algorithms.

Source: Zee Entertainment — Data Scientist

I would choose the representation based on the task.

For a classical ML baseline:

Text
 ↓
Tokenization
 ↓
TF-IDF / BoW
 ↓
ML classifier

For semantic tasks:

Text
 ↓
Embedding model
 ↓
Dense vector
 ↓
Similarity / classifier / downstream model

For modern NLP:

Text
 ↓
Transformer tokenizer
 ↓
Transformer model
 ↓
Contextual representation

Source: ZS Associates — Data Science Associate

Suppose we represent a sentence by averaging all word vectors.

The result loses:

• Word order
• Syntactic structure
• Token-level context
• Some information about which words are important

For example:

"dog bites man"

and

"man bites dog"

can produce similar averaged representations.

Better approaches include:

• TF-IDF-weighted embeddings
• RNN/LSTM
• CNN
• Transformer encoders
• Sentence Transformer models

Source: Zee Entertainment / Zycus interview reports

Word embeddings represent words as dense numerical vectors.

The vectors are learned so that words appearing in similar contexts tend to have related representations.

Examples:

king
queen
doctor
hospital

Unlike one-hot vectors, embeddings can capture relationships between words.

Source: ZS Associates / Zycus

Word2Vec is a neural approach for learning word embeddings from context.

It has two major architectures:

CBOW
Skip-gram

CBOW predicts a target word from surrounding context.

Skip-gram predicts surrounding context words from a target word.

Source: Zycus — AI/Machine Learning Engineer

Skip-gram takes a target word and tries to predict nearby context words.

Example:

"The cat sat on the mat"

For target:

cat

the model may learn to predict nearby words such as:

the
sat

The model learns useful word representations as a consequence of this prediction task.

Source: Zycus — AI/Machine Learning Engineer

CBOW stands for Continuous Bag of Words.

It predicts the target word from surrounding context.

Example:

The cat ___ on the mat

The surrounding words are used to predict:

sat

CBOW and Skip-gram are two different Word2Vec training objectives.

Source: Zycus — AI/Machine Learning Engineer

CBOW:
Context → Target

Skip-gram:
Target → Context

CBOW generally trains faster because it predicts a target from context.

Skip-gram can be useful for learning representations of less frequent words because it generates multiple context-prediction examples.

Source: ZS Associates — Data Science Associate

It depends on the domain and data.

Pretrained embeddings

Advantages:

• Already trained on large corpora.
• Useful when my dataset is small.
• Can provide stronger general semantic representations.

Training on domain data

Advantages:

• Can capture domain-specific vocabulary and usage.
• Useful when the application contains specialized terminology.

I would compare the alternatives using validation performance rather than assuming pretrained embeddings are always better.

Source: ZS Associates — Data Science Associate

Not necessarily.

Performance depends on:

• Domain similarity
• Corpus size
• Vocabulary
• Task
• Embedding quality
• Amount of training data

For a specialized domain, an embedding trained on domain-specific text can sometimes capture terminology better.

The correct approach is to evaluate both alternatives.

Source: Zycus — AI/Machine Learning Engineer

Both learn dense word representations but use different training approaches.

Word2Vec

Learns embeddings through prediction tasks:

CBOW
Skip-gram

GloVe

Uses global word co-occurrence statistics to learn vectors.

A useful interview summary is:

Word2Vec → predictive context-based training
GloVe    → global co-occurrence statistics

Source: ZS Associates — Data Science Associate

I would compare models systematically.

For example:

TF-IDF
   ↓
Logistic Regression
   ↓
Naive Bayes
   ↓
SVM

Then compare them using appropriate metrics.

For an imbalanced classification problem, I would not rely only on accuracy.

I would examine:

• Precision
• Recall
• F1-score
• Confusion matrix
• Class-wise performance

Source: ZS Associates — Data Science Associate

First, I would measure the class distribution.

Possible approaches include:

• Class weights
• Oversampling
• Undersampling
• Data augmentation
• Threshold adjustment

I would evaluate using metrics appropriate for imbalance, such as:

Precision
Recall
F1-score
PR-AUC

I would also inspect per-class performance.

Source: ZS Associates — Data Science Associate

I would investigate the problem systematically:

Data quality
↓
Label quality
↓
Class imbalance
↓
Preprocessing
↓
Feature engineering
↓
Representation
↓
Model
↓
Hyperparameters
↓
Error analysis

I would inspect false positives and false negatives to understand why the model fails.

Source: ZS Associates — Data Science Associate

Potentially, but not automatically.

Deep learning may help when:

• There is enough training data.
• The task depends on complex context.
• Traditional representations are insufficient.
• We need learned semantic representations.

Possible models include:

• CNN
• RNN
• LSTM
• Transformer

I would compare against a strong classical baseline rather than assuming deep learning is always better.

Source: JPMorganChase — NLP Summer

An RNN processes a sequence one element at a time and maintains a hidden state.

Conceptually:

x1 → RNN → h1
x2 → RNN → h2
x3 → RNN → h3

The hidden state carries information from previous tokens.

This makes RNNs suitable for sequential data.

Source: KnowDis — Data Scientist

Major issues include:

• Vanishing gradients
• Exploding gradients
• Difficulty learning long-term dependencies
• Sequential computation

LSTM and GRU architectures were designed to improve the handling of long-term dependencies.

Source: KnowDis — Data Scientist

Possible techniques include:

• Gradient clipping for exploding gradients.
• Better initialization.
• Appropriate activation functions.
• Architectural changes such as gated mechanisms.
• Careful sequence handling and truncated backpropagation.

For long-term dependencies, LSTM/GRU or Transformers are often more practical.

Source: Zee Entertainment / Zycus interview reports

LSTM stands for Long Short-Term Memory.

An LSTM contains:

Forget Gate
Input Gate
Output Gate
Cell State
Hidden State

The gates control what information should be:

forgotten
stored
exposed

This helps reduce the long-term dependency problems of a vanilla RNN.

Source: JPMorganChase — NLP Summer

A Transformer processes tokens using attention rather than recurrence.

A simplified Transformer flow is:

Input Tokens
     ↓
Token Embeddings
     +
Positional Information
     ↓
Self-Attention
     ↓
Feed Forward Network
     ↓
Repeated Transformer Layers
     ↓
Output Representation

The architecture enables much more parallel computation than traditional RNNs.

Source: JPMorganChase — NLP Summer

Self-attention allows each token to determine how much attention it should give to other tokens in the sequence.

It uses:

Query
Key
Value

The standard attention calculation is:

Attention(Q,K,V)
=
softmax(QKᵀ / √dₖ)V

This allows the model to build contextual representations.

Source: Level AI — Machine Learning Engineer - NLP

Self-attention calculates attention across the sequence.

Multi-head attention runs multiple attention heads in parallel.

Each head can learn different relationships.

Conceptually:

Input
 ↓
Head 1 → relationship type 1
Head 2 → relationship type 2
Head 3 → relationship type 3
...
 ↓
Concatenate
 ↓
Linear projection

This allows the model to capture different types of relationships simultaneously.

Source: KnowDis / Level AI

Transformers do not inherently process tokens sequentially like RNNs.

Therefore, the model needs information about token positions.

For example:

dog bites man

is different from:

man bites dog

Positional information helps the model distinguish these orders.

Source: KnowDis — Data Scientist

Important components include:

Token embeddings
Positional information
Multi-head attention
Feed-forward network
Residual connections
Layer normalization

The exact structure differs between encoder-only, decoder-only and encoder-decoder Transformer architectures.

Source: KnowDis — Data Scientist

Encoder

Primarily builds contextual representations from input tokens.

Used in architectures such as:

BERT

Decoder

Generates tokens autoregressively in decoder-only models such as:

GPT-style models

Encoder-decoder

Uses an encoder to process input and a decoder to generate output.

Examples include many sequence-to-sequence Transformer architectures.

Source: KnowDis — Data Scientist

A decoder used for autoregressive generation applies a causal mask so that a token cannot attend to future tokens.

For example:

I love machine learning

When predicting:

machine

the model can use:

I love

but should not use future tokens:

learning

This prevents information leakage from future positions during training.

Source: KnowDis — Data Scientist

An autoregressive language model predicts the next token based on previous tokens.

Example:

Input:
"I love"

Prediction:
"machine"

Then:
"I love machine"

Prediction:
"learning"

This process continues token by token.

Decoder-only Transformer models commonly use this objective.

Source: Level AI — Machine Learning Engineer - NLP

Sentence Transformers are models designed to produce dense vector representations for sentences or larger text units.

They are commonly used for:

• Semantic similarity
• Semantic search
• Clustering
• Retrieval
• Duplicate detection

For example:

"I bought a car."
"I purchased an automobile."

can receive similar sentence embeddings despite different wording.

Source: IBM — NLP Researcher; Level AI — Machine Learning Engineer - NLP

I would consider the complete pipeline:

Documents
   ↓
Parsing
   ↓
Chunking
   ↓
Embedding
   ↓
Vector / Hybrid Index
   ↓
Retrieval
   ↓
Optional reranking
   ↓
Context construction
   ↓
LLM
   ↓
Answer

Important design considerations include:

• Chunk size
• Chunk overlap
• Embedding model
• Retrieval strategy
• Top-k
• Metadata filtering
• Reranking
• Context limits
• Citation/grounding
• Evaluation
• Latency
• Cost

Source: KnowDis — Machine Learning Engineer interview

A practical search system can contain:

User Query
   ↓
Query preprocessing
   ↓
Candidate retrieval
   ↓
Keyword / BM25 / vector search
   ↓
Optional reranking
   ↓
Top results

For semantic search, embeddings can be used to retrieve documents based on meaning rather than exact keyword overlap.

A production system may combine:

Keyword search + semantic/vector search

Source: ZS Associates — Data Science Associate

I would approach it as an end-to-end ML problem:

1. Understand the business objective
2. Inspect and label data
3. Explore the data
4. Check class distribution
5. Clean and preprocess text
6. Engineer useful features
7. Split data correctly
8. Build a baseline
9. Compare TF-IDF / n-grams / embeddings
10. Train multiple models
11. Evaluate using appropriate metrics
12. Perform error analysis
13. Tune the best approach
14. Deploy
15. Monitor and retrain

The important part is explaining why each decision was made.

Answer: [TODO]

Why remove stopwords in the first place: words like "the," "is," "a," "and" occur in almost every document regardless of class/topic, so they add little discriminative value for tasks like topic classification — removing them shrinks the vocabulary, reduces noise, and speeds up training.

Why you should not always remove them: the classic counter-example is negation-sensitive tasks:

"The movie was not good at all."
→ after naive stopword removal: "movie good"

The review is negative, but stripping "not," "was," "at," "all" flips the apparent sentiment to positive — this is why stopword removal has to be a task-dependent decision, not a blanket step.

Remove stopwords when the task cares about topic/frequency signal, not exact wording or word order:

Do NOT remove stopwords when meaning depends on function words, negation, or word order:

One-line interview summary: "Remove stopwords for frequency/topic-based classical pipelines where function words are pure noise; keep them for anything sequence-, negation-, or grammar-sensitive — and virtually always keep them when feeding a pretrained Transformer, since removal creates a mismatch with how that model was pretrained."

Given

raw_text = """
Heyyyy!!! I justtt bought this product from @Amazon on 15/09/2026 😍. TBH, it's sooo goodddd!!! The price was ₹1,999.99 & I got 20% OFF!!! But the delivery took 3 days... idk why they were sooo slow :/ Plz check ur order carefully b4 buying. I can't say it's 100% perfect, but I'd definitely recommend it!!! BTW, customer service was gr8 & they replied ASAP. This product is MUCHHH better than the previous one!!! #AmazingDeal #WorthIt
"""

Task

Perform the following NLP preprocessing steps:

• Remove mentions such as @Amazon
• Remove hashtags such as #AmazingDeal, #WorthIt
• Convert text to lowercase
• Expand contractions (it's → it is, can't → cannot, I'd → I would)
• Normalize repeated characters (Heyyyy → Hey, justtt → just, sooo → so, goodddd → good, MUCHHH → much)
• Expand common chat abbreviations (TBH → to be honest, idk → I do not know, Plz → please, ur → your, b4 → before, BTW → by the way, gr8 → great, ASAP → as soon as possible)
• Remove emojis
• Remove punctuation and special characters
• Remove numbers
• Normalize extra spaces
• Tokenize the final text into words

Expected clean text

clean_text = """hey i just bought this product on it is to be honest it is so good the price was and i got off but the delivery took days i do not know why they were so slow please check your order carefully before buying i cannot say it is perfect but i would definitely recommend it by the way customer service was great and they replied as soon as possible this product is much better than the previous one"""

Approach — order matters

1. Expand contractions and chat abbreviations first, while punctuation like apostrophes and word boundaries are still intact.
2. Remove mentions (@word) and hashtags (#word) before stripping punctuation, since @/# are the anchor characters being matched.
3. Remove emojis.
4. Normalize repeated characters (sooo → so, MUCHHH → much) before lowercasing/punctuation removal, while the run-length pattern is still clean.
5. Lowercase.
6. Remove currency amounts, dates, percentages and remaining numbers.
7. Remove punctuation/special characters (keep letters and spaces only).
8. Collapse extra whitespace.
9. Tokenize.

Solution

import re

def clean_text_fn(text: str) -> str:
    # 1. Expand contractions (before punctuation is stripped)
    contractions = {
        r"\bit's\b": "it is",
        r"\bcan't\b": "cannot",
        r"\bi'd\b": "i would",
        r"\bi'm\b": "i am",
        r"\bwon't\b": "will not",
        r"\bdon't\b": "do not",
    }
    for pattern, repl in contractions.items():
        text = re.sub(pattern, repl, text, flags=re.IGNORECASE)

    # 2. Expand common chat abbreviations (word-boundary matched, case-insensitive)
    abbreviations = {
        r"\btbh\b": "to be honest",
        r"\bidk\b": "i do not know",
        r"\bplz\b": "please",
        r"\bur\b": "your",
        r"\bb4\b": "before",
        r"\bbtw\b": "by the way",
        r"\bgr8\b": "great",
        r"\basap\b": "as soon as possible",
    }
    for pattern, repl in abbreviations.items():
        text = re.sub(pattern, repl, text, flags=re.IGNORECASE)

    # 3. Remove mentions and hashtags
    text = re.sub(r"@\w+", "", text)
    text = re.sub(r"#\w+", "", text)

    # 4. Remove emojis (broad Unicode emoji ranges)
    emoji_pattern = re.compile(
        "["
        "\U0001F300-\U0001FAFF"  # symbols & pictographs, emoticons, transport, supplemental
        "\U00002700-\U000027BF"  # dingbats
        "\U0001F1E0-\U0001F1FF"  # flags
        "]+",
        flags=re.UNICODE,
    )
    text = emoji_pattern.sub("", text)

    # 5. Normalize repeated characters: 3+ repeats of a letter -> collapse to 1
    #    ("sooo" -> "so", "goodddd" -> "good", "MUCHHH" -> "much")
    text = re.sub(r"(.)\1{2,}", r"\1", text)

    # 6. Lowercase
    text = text.lower()

    # 7. Remove currency amounts / dates / percentages / remaining numbers
    text = re.sub(r"[₹$€]\s?[\d,]+\.?\d*", " ", text)   # currency amounts
    text = re.sub(r"\d+/\d+/\d+", " ", text)             # dates
    text = re.sub(r"\d+%", " ", text)                    # percentages
    text = re.sub(r"\d+", " ", text)                     # remaining standalone numbers

    # 8. Remove punctuation / special characters (keep letters and spaces only)
    text = re.sub(r"[^a-z\s]", " ", text)

    # 9. Collapse extra whitespace
    text = re.sub(r"\s+", " ", text).strip()

    return text


def tokenize(text: str) -> list[str]:
    return text.split()


cleaned = clean_text_fn(raw_text)
tokens = tokenize(cleaned)

print(cleaned)
print(tokens)

Output (matches clean_text above)

hey i just bought this product on it is to be honest it is so good the price was and i got off but the delivery took days i do not know why they were so slow please check your order carefully before buying i cannot say it is perfect but i would definitely recommend it by the way customer service was great and they replied as soon as possible this product is much better than the previous one

Common follow-up questions after this task

Why expand contractions/abbreviations before removing punctuation instead of after?
Why normalize repeated characters before lowercasing?
How would this regex distinguish "sooo" (elongation) from a genuine word like "wooord" if one existed?
How would you scale this to millions of rows? (vectorized string ops, spaCy pipe, multiprocessing)
How would you adapt this for a language where \b word boundaries don't behave the same way (e.g. Chinese, Thai)?
What would you change if this text were feeding into a Transformer instead of TF-IDF?
   (Direction: keep more of the original text — a cased Transformer tokenizer handles
    casing/subwords itself, and over-aggressive cleaning can remove signal the
    pretrained model would otherwise use.)
How would you unit-test this function?