HashSet for Duplicate / Existence Check
A HashSet answers one question — 'have I seen this before?' — in O(1) time. Mastering this simple structure unlocks a surprising range of problems: duplicate detection, existence lookup, the elegant Longest Consecutive Sequence, and cycle detection via the Floyd variant.
Table of contents
- The problem with scanning
- The bouncer with a stamp
- The one idea to remember
- The skeleton
- Variant 1 — Contains Duplicate (LC #217)
- Variant 2 — Contains Duplicate within k distance (LC #219)
- Variant 3 — Longest Consecutive Sequence (LC #128)
- The key insight: only start counting from sequence heads
- Frame-by-frame for [100, 4, 200, 1, 3, 2]
- Why is this O(n) and not O(n²)?
- Set vs. dict — when to use which
- Cycle detection via HashSet (Floyd alternative)
- Where to spot this pattern
- Common traps
- Complexity
- Say it like a pro (interview one-liner)
- Check yourself
- Practice problems
The problem with scanning
You have a list of 10 000 numbers and want to know if any two are the same. The obvious approach: for each number, scan the rest of the list to see if it appears again. That is O(n²) comparisons.
There is a faster way. You don't need to find which pair is a duplicate — only whether one exists. For that, you need a structure that answers "have I seen this?" in constant time.
The bouncer with a stamp
A real-life story
A nightclub bouncer checks ID at the door. As each guest enters, he stamps their wrist. When a guest approaches, he asks: "Do you have a stamp?"
- No stamp → new guest, let them in, stamp them.
- Stamp found → already here → duplicate.
The bouncer doesn't need to remember the names or arrival order of 10 000 guests — he only needs to check for the stamp. A HashSet is that stamp system: add an item, check if it's already there, in O(1) time.
The one idea to remember
The entire pattern in one sentence
A HashSet stores only keys (no values), supports O(1) add and O(1) lookup, and answers "have I seen this element before?" — turning O(n²) pair-scanning into a single O(n) pass.
The skeleton
seen = set()
for x in arr:
if x in seen: # O(1) lookup
# x is a duplicate — act on it
seen.add(x) # O(1) insert
That's the entire pattern. Everything else is a variation on where to add, where to query, and what "act on it" means.
Variant 1 — Contains Duplicate (LC #217)
Does the array contain any duplicate?
def containsDuplicate(nums: list[int]) -> bool:
seen = set()
for num in nums:
if num in seen:
return True # duplicate found → short-circuit
seen.add(num)
return False
Pause & think
Why not just use len(nums) != len(set(nums))? It works, but it always scans the full array. The explicit loop above returns immediately on the first duplicate. For a 1M-element array where the first two elements are identical, the early-return version does 2 operations; the set(nums) version does 1M.
Variant 2 — Contains Duplicate within k distance (LC #219)
Does any duplicate exist where the two copies are at most k index positions apart?
def containsDuplicateII(nums: list[int], k: int) -> bool:
window = set()
for i, num in enumerate(nums):
if num in window:
return True
window.add(num)
if len(window) > k:
window.remove(nums[i - k]) # slide: remove element k steps back
return False
This is a sliding-window HashSet. The window always holds at most k elements. Checking membership against this bounded set answers "does a copy within k distance exist?"
Variant 3 — Longest Consecutive Sequence (LC #128)
This is the "aha" problem of the HashSet chapter.
Problem: given an unsorted array, find the length of the longest sequence of consecutive integers (e.g., [100, 4, 200, 1, 3, 2] → 4 because 1, 2, 3, 4).
Naive approach: sort, then scan. O(n log n).
HashSet approach: O(n). The trick is elegant.
The key insight: only start counting from sequence heads
A number x is a sequence head if x - 1 is not in the set. If x - 1 exists, then x is the middle of a sequence that was already (or will be) counted from the earlier head. Starting from the head prevents counting the same sequence multiple times.
def longestConsecutive(nums: list[int]) -> int:
num_set = set(nums) # O(n) build
best = 0
for x in num_set:
if x - 1 not in num_set: # x is a sequence head
length = 1
while x + length in num_set:
length += 1
best = max(best, length)
return best
Frame-by-frame for [100, 4, 200, 1, 3, 2]
num_set = {1, 2, 3, 4, 100, 200}
x = 100 → is 99 in set? No → head. Count: 100, 101? No → length = 1.
x = 4 → is 3 in set? Yes → not a head. Skip.
x = 200 → is 199 in set? No → head. Count: 200, 201? No → length = 1.
x = 1 → is 0 in set? No → head. Count: 1, 2 ✓, 3 ✓, 4 ✓, 5? No → length = 4.
x = 3 → is 2 in set? Yes → not a head. Skip.
x = 2 → is 1 in set? Yes → not a head. Skip.
best = 4 ✅
Why is this O(n) and not O(n²)?
The while loop extends from a head and visits each element of the sequence once. Across all heads, the total number of while iterations equals the total number of elements — O(n). The outer for loop is also O(n). Combined: O(n).
Pause & think
What if you iterated over nums (the original list with duplicates) instead of num_set? Would the algorithm still be correct? Would it still be O(n)?
Answer
Correct: yes. Duplicate values are not sequence heads (the duplicate's x - 1 may or may not be in the set; if it is, skip; if it isn't, you'd count the same sequence twice). O(n): not guaranteed. If the array has many duplicates all being heads, you'd recount the same sequence many times. Iterating over num_set guarantees each distinct value is visited once.
Set vs. dict — when to use which
| Need | Structure | Why |
|---|---|---|
| Only "have I seen this?" | set | No value needed; smaller memory footprint |
| "Have I seen this, and what was its value/index?" | dict | Map key → additional data |
| Existence + order | Sorted set / sortedcontainers.SortedList | Ordered membership in O(log n) |
| Count occurrences | dict / Counter | Set can't count; dict maps key → frequency |
A common beginner mistake: using a dict with dummy values ({x: True}) when a set is all you need. Use the simplest structure.
Cycle detection via HashSet (Floyd alternative)
If you want to detect a cycle in a linked list or sequence of jumps, HashSet gives you a simple O(n) space alternative to Floyd's two-pointer algorithm:
def hasCycle(head):
seen = set()
node = head
while node:
if id(node) in seen:
return True
seen.add(id(node))
node = node.next
return False
Floyd's algorithm is O(1) space and generally preferred in interviews, but the HashSet version is O(1) time per step and easier to adapt for "which node is the cycle entry?"
Where to spot this pattern
Trigger words:
- "contains duplicate"
- "any repeated element"
- "check existence" or "has this appeared before"
- "longest consecutive" or "consecutive integers"
- "visited" tracking during graph/path traversal
- "intersection" or "union" of two arrays (use set operations)
5 disguises:
- Contains Duplicate (LC #217): direct bouncer pattern.
- Contains Duplicate II (LC #219): sliding window of size k.
- Longest Consecutive Sequence (LC #128): head-only counting.
- Happy Number (LC #202): cycle detection — has a number appeared in the digit-sum sequence?
- Intersection of Two Arrays (LC #349): convert both to sets,
set1 & set2.
Common traps
Watch out for these
- Iterating over the original list (with duplicates) in the consecutive sequence problem. Duplicates are visited multiple times and can inflate the inner while loop. Always iterate over
num_set. - Forgetting to build the full set before the main loop. In "Longest Consecutive Sequence," the existence check
x + length in num_setrequires the entire set to be pre-built. A single combined pass (build set and check simultaneously) can miss elements not yet inserted. - Using a dict when you need a set. Extra overhead, harder to read.
- Assuming sets are ordered. Python's
setis unordered. If you need sorted iteration, usesorted(num_set)or a sorted data structure.
Complexity
| Operation | Time | Space |
|---|---|---|
| Add element | O(1) amortised | — |
| Lookup element | O(1) amortised | — |
| Build set of n elements | O(n) | O(n) |
| Contains Duplicate scan | O(n) | O(n) |
| Longest Consecutive Sequence | O(n) | O(n) |
Say it like a pro (interview one-liner)
"I'll use a HashSet to answer 'have I seen this?' in O(1) time per element, giving O(n) total for a single pass. For Longest Consecutive, I extend only from sequence heads — elements where
x - 1is absent — so each element is visited by the inner loop at most once, keeping the total to O(n)."
Remember this forever
HashSet — Bouncer Pattern
seen = set()
for x in arr:
if x in seen: # O(1) — have I seen this?
# duplicate / cycle / hit
seen.add(x) # O(1) — stamp it
Longest Consecutive Sequence trick:
num_set = set(nums)
for x in num_set:
if x - 1 not in num_set: # sequence head only
count from x upward with while x+length in num_set
Trigger: duplicate check, existence query, "visited" tracking, consecutive integers.
Trap: in Longest Consecutive, iterate over the SET (not the list) and build the full set first.
Check yourself
Why must you build the full `num_set` before iterating in the Longest Consecutive Sequence solution?
The inner while x + length in num_set checks whether the next integer exists anywhere in the array. If you build the set and check simultaneously, elements that appear later in the array haven't been inserted yet — you'd stop sequences prematurely. Building the set first ensures every element is available for lookup before any counting begins.
What is the "head-only" trick and why does it keep the algorithm O(n)?
A sequence head is any number x where x - 1 is not in the set — meaning x starts a new consecutive run. If x - 1 exists, x is already inside some other sequence; counting from it would recount the prefix. By skipping non-heads, each sequence is counted exactly once from its starting point. The total steps across all inner while loops equals the total number of distinct elements — O(n).
In Variant 2 (k-distance duplicate), why is `window.remove(nums[i - k])` correct and not `window.remove(nums[i - k - 1])`?
The window must contain all elements with indices in the range [i - k, i] — a window of exactly k+1 elements (or fewer). When the window already has k elements (indices i-k through i-1) and we add element at index i, the window has k+1 elements. We then remove the element at index i - k to shrink back to size k, ready for the next iteration. The element at i - k is the one that would be k+1 positions behind the new index i+1, so removing it keeps the k-distance invariant.
Practice problems
| Problem | Difficulty | What to notice | Link |
|---|---|---|---|
| Contains Duplicate | Easy | Direct bouncer; short-circuit on first hit | LC #217 |
| Contains Duplicate II | Easy | Sliding window set of size k | LC #219 |
| Longest Consecutive Sequence | Medium | Head-only counting; iterate over set not list | LC #128 |
| Happy Number | Easy | Cycle detection via set; stop when 1 or revisit | LC #202 |
| Intersection of Two Arrays | Easy | set(a) & set(b) — let Python do the work | LC #349 |
| Single Number | Easy | Interesting contrast: XOR trick beats HashSet in space | LC #136 |
Next up: Custom Hash Design — when you need to hash composite keys (pairs, tuples, strings with structure), and how to design a hash function that avoids collisions.