Benchmarks

Performance Benchmarks: Generic vs Reflection

Last Updated: 2026-01-20

Quick Summary

We added 38 generic assertion functions to testify v2, providing type-safe alternatives to reflection-based assertions. While the primary goal was compile-time type safety, comprehensive benchmarking revealed an unexpected bonus: dramatic performance improvements.

Key Results:

• Type Safety: Catch errors when writing tests, not when running them
• Performance: 1.2x to 81x faster depending on the operation
• Memory: Up to 99% reduction in allocations for collection operations
• Zero Downside: Generic variants are always as fast or faster

The Type Safety Story

The main reason for adding generics wasn’t performance—it was catching bugs earlier.

Before: Runtime Surprises

// Compiles fine, fails mysteriously at runtime
assert.Equal(t, []int{1, 2}, []string{"a", "b"})
assert.ElementsMatch(t, userIDs, orderIDs)  // Wrong comparison!

After: Compile-Time Safety

// Compiler catches the error immediately
assert.EqualT(t, []int{1, 2}, []string{"a", "b"})  // ❌ Compile error!
assert.ElementsMatchT(t, userIDs, orderIDs)        // ❌ Type mismatch!

Real-world benefit: When refactoring changes a type from []int to []string, generic assertions immediately flag all broken tests. Reflection-based assertions compile but fail during test runs—or worse, pass with wrong comparisons.

Performance Highlights

While type safety was the goal, benchmarking revealed impressive performance gains across all domains.

🏆 Collection Operations: The Big Winner

Why it matters: Collection operations are common in tests. ElementsMatchT is up to 81x faster and uses 99% less memory for large slices.

⚡ Comparison Operations

Why it matters: Direct operator usage (>, <) eliminates reflection overhead and boxing.

✓ Equality Checks

Why it matters: Equality checks are the most common assertion. 10x speedup adds up quickly.

📊 Ordering Operations

Why it matters: Ordering checks iterate over collections. Generics eliminate per-element reflection overhead.

What This Means for You

Always Prefer Generic Variants

When a generic variant is available (functions ending in T), use it:

// OLD: Reflection-based
assert.Equal(t, 42, result)
assert.Greater(t, count, 0)
assert.ElementsMatch(t, expected, actual)

// NEW: Type-safe + faster
assert.EqualT(t, 42, result)           // 13x faster + compile-time safety
assert.GreaterT(t, count, 0)           // 16x faster + compile-time safety
assert.ElementsMatchT(t, expected, actual)  // 21-81x faster + compile-time safety

Type Safety Catches Real Bugs

Example 1: Refactoring Safety

// Your test
assert.ElementsMatchT(t, userIDs, orderIDs)

// Someone changes OrderID type
type OrderID string  // Was: int

// Generic version: Compiler catches the error
assert.ElementsMatchT(t, userIDs, orderIDs)  // ❌ Compile error!

// Reflection version: Test compiles, fails mysteriously
assert.ElementsMatch(t, userIDs, orderIDs)   // ✓ Compiles, ✗ Wrong at runtime

Example 2: IDE Assistance

Generic variants enable IDE autocomplete to suggest only correctly-typed variables, preventing copy-paste errors.

When to Use Reflection Variants

Keep reflection-based assertions for:

• Intentional cross-type comparisons (e.g., int vs int64 with EqualValues)
• Heterogeneous collections ([]any)
• Dynamic type scenarios where compile-time type is unknown
• Backward compatibility with existing tests

Performance Tiers

Tier 1: Dramatic Improvements (10x+)

These operations see the biggest speedups because reflection overhead dominates:

• ElementsMatch: 21-81x (scales with collection size)
• Equal/NotEqual: 10-13x
• Comparison operators: 10-22x
• Type checks: 9-11x

Tier 2: Significant Improvements (3-10x)

Solid gains from eliminating reflection:

• Ordering checks: 6.5-9.5x
• Collection operations: 7.5-43x

Tier 3: Modest Improvements (1.2-3x)

Operations already optimized see smaller gains:

• Same/NotSame: 1.5-2x
• Numeric comparisons: 1.2-1.5x
• Boolean checks: 2x

Tier 4: Comparable Performance

Operations where expensive processing dominates:

• JSONEq: JSON parsing dominates (marginal difference)
• Regexp: Regex compilation dominates (marginal difference)

Key insight: Even when performance gains are modest, type safety alone justifies using generic variants.

Real Benchmark Results

ElementsMatch: The Star Performer

BenchmarkElementsMatch/reflect/10-16     3259 ns/op     568 B/op      67 allocs/op
BenchmarkElementsMatch/generic/10-16      154 ns/op     320 B/op       2 allocs/op
                                          ↑ 21x faster

BenchmarkElementsMatch/reflect/100-16   291692 ns/op   41360 B/op    5153 allocs/op
BenchmarkElementsMatch/generic/100-16     7429 ns/op    3696 B/op       3 allocs/op
                                          ↑ 39x faster

BenchmarkElementsMatch/reflect/1000-16  25.5 ms/op     4.0 MB/op    501503 allocs/op
BenchmarkElementsMatch/generic/1000-16   316 µs/op     33 KB/op         3 allocs/op
                                          ↑ 81x faster   ↑ 99% less memory

Comparison Operations

BenchmarkGreater/reflect/int-16         139.1 ns/op      34 B/op       1 allocs/op
BenchmarkGreater/generic/int-16          17.9 ns/op       0 B/op       0 allocs/op
                                         ↑ 7.8x faster

BenchmarkPositive/reflect/int-16        121.5 ns/op      26 B/op       1 allocs/op
BenchmarkPositive/generic/int-16          7.6 ns/op       0 B/op       0 allocs/op
                                         ↑ 16x faster

Equality Checks

BenchmarkEqual/reflect/int-16            44.8 ns/op       0 B/op       0 allocs/op
BenchmarkEqual/generic/int-16             3.5 ns/op       0 B/op       0 allocs/op
                                         ↑ 13x faster

BenchmarkEqual/reflect/string-16         34.8 ns/op       0 B/op       0 allocs/op
BenchmarkEqual/generic/string-16          4.1 ns/op       0 B/op       0 allocs/op
                                         ↑ 8.5x faster

Why These Numbers Matter

1. Allocation Elimination

The most dramatic speedups come from eliminating allocations entirely:

• ElementsMatch: 501,503 → 3 allocations (1000 elements)
• All comparisons: 1 → 0 allocations
• Ordering checks: 4-11 → 0 allocations

Less allocation pressure means faster execution and reduced GC overhead.

2. Superlinear Scaling

ElementsMatch’s O(n²) complexity amplifies the benefits:

• 10 elements: 21x faster
• 100 elements: 39x faster
• 1000 elements: 81x faster

The speedup increases with collection size.

3. Cumulative Impact

If your test suite uses assertions thousands of times:

• 10x speedup per assertion = significantly faster test runs
• Especially impactful in CI/CD pipelines

Migration Guide

Step 1: Identify Generic-Capable Assertions

Look for these common assertions in your tests:

• Equal, NotEqual → EqualT, NotEqualT
• Greater, Less, Positive, Negative → GreaterT, LessT, PositiveT, NegativeT
• Contains, ElementsMatch, Subset → ContainsT, ElementsMatchT, SubsetT
• IsIncreasing, IsDecreasing → IsIncreasingT, IsDecreasingT
• IsOfType → IsOfTypeT (eliminates need for dummy values!)

Step 2: Add Type Parameters

// Before
assert.Equal(t, expected, actual)

// After: Add T suffix, compiler checks types
assert.EqualT(t, expected, actual)

Step 3: Fix Type Mismatches

The compiler will now catch type errors:

// This will now fail to compile
assert.EqualT(t, int64(42), int32(42))

// Fix by using the same type
assert.EqualT(t, int64(42), int64(actual))

// Or use reflection-based Equal for intentional cross-type comparison
assert.Equal(t, int64(42), int32(42))  // Uses reflection, still works

Conclusion

Generic assertions deliver two major benefits:

1. Type Safety (Primary Goal): Catch errors when writing tests

   • Compiler catches type mismatches immediately
   • IDE autocomplete guides to correct types
   • Refactoring safety: broken tests identified at compile time

2. Performance (Unexpected Bonus): 1.2-81x faster

   • Zero allocation overhead for most operations
   • Dramatic gains for collection operations
   • Cumulative benefits across large test suites

Recommendation: Prefer generic variants (*T functions) wherever available. The type safety alone justifies the switch; the performance improvement is a bonus.

The Bottom Line

// What we wanted: Catch this when writing tests
assert.ElementsMatchT(t, []int{1,2}, []string{"a","b"})  // ❌ Compiler error

// What we got as bonus: 81x faster when types match
assert.ElementsMatchT(t, []int{1,2}, []int{2,1})  // ✓ Type safe AND blazing fast

The performance improvements validate the design choice, but type safety was always the goal.

Running Your Own Benchmarks

# Run all benchmarks
go test -run=^$ -bench=. -benchmem ./internal/assertions

# Specific domain (equality, comparison, collection, etc.)
go test -run=^$ -bench='Benchmark(Equal|Same)' -benchmem ./internal/assertions

# Compare specific function
go test -run=^$ -bench='BenchmarkElementsMatch' -benchmem ./internal/assertions

Coverage

38 generic functions benchmarked across 10 domains:

• Boolean (2): TrueT, FalseT
• Collection (12): StringContainsT, SliceContainsT, MapContainsT, SeqContainsT, ElementsMatchT, SliceSubsetT, and negative variants
• Comparison (6): GreaterT, LessT, GreaterOrEqualT, LessOrEqualT, PositiveT, NegativeT
• Equality (4): EqualT, NotEqualT, SameT, NotSameT
• JSON (1): JSONEqT
• Number (2): InDeltaT, InEpsilonT
• Ordering (6): IsIncreasingT, IsDecreasingT, IsNonIncreasingT, IsNonDecreasingT, SortedT, NotSortedT
• String (2): RegexpT, NotRegexpT
• Type (2): IsOfTypeT, IsNotOfTypeT
• YAML (1): YAMLEqT (benchmarked separately in enable/yaml module)