Catalypt LogoCatalypt.ai

Industry Focus

Developer Options

Resources

Back to Blog

Pair Programming With AI: The Good, The Bad, The Hallucinated

May 25, 2024 Josh Butler Workflows

"It's like having a senior developer always available!" That's what they told me about AI pair programming. What they didn't mention: this senior developer sometimes forgets what language we're using and occasionally invents entire frameworks.

After a year of daily AI pairing, here's the real story.

The Honeymoon Phase

Week 1 with Copilot/Claude/Cursor was magical:

  • Autocompleting entire functions!
  • Writing perfect boilerplate!
  • Suggesting variable names that make sense!
  • I'm 10x more productive!

Week 2 reality check:

  • Autocompleted a function that calls non-existent APIs
  • Suggested we import from 'react-native' in a web app
  • Confidently used a Python decorator in JavaScript
  • I'm now debugging AI mistakes at 2x speed

The Hallucination Hall of Fame

Real AI suggestions from my pairing sessions:

// Me: "Import the user authentication module"
// AI: 
import { authenticate } from '@facebook/advanced-auth-suite';

// That package doesn't exist. Never has.

// Me: Starting a React component
// AI: Autocompletes with Vue syntax
<template>
  <div>{{ message }}</div>
</template>

// We're... we're in a .jsx file

// Me: "Connect to PostgreSQL"
// AI: 
const mysql = require('mysql');

// Close enough? 🤷

When AI Pair Programming Shines

1. Boilerplate and Patterns

// Type "useState" and it knows:
const [loading, setLoading] = useState(false);
const [error, setError] = useState(null);
const [data, setData] = useState(null);

// Saves 30 seconds every time

2. Test Writing

// Write one test, AI writes similar ones
it('should handle empty array', () => { ... });
// AI suggests:
it('should handle null input', () => { ... });
it('should handle undefined input', () => { ... });
it('should handle array with one item', () => { ... });

3. Regex and Algorithms
"Write a regex for email validation" - AI is faster than Stack Overflow
"Implement quicksort" - Perfect every time
"Parse this weird date format" - Actually helpful

When It Goes Off the Rails

The Context Confusion

// File: frontend/components/UserList.jsx
// AI keeps suggesting Node.js APIs:
const fs = require('fs');
const users = JSON.parse(fs.readFileSync('./users.json'));

// BRO WE'RE IN THE BROWSER

The Overeager Helper

// Me: Types "function"
// AI: Autocompletes entire 200-line implementation
// Me: I just wanted to write "function getName()"
// AI: Too late, here's a complete user management system

The Inconsistent Partner

// Line 10: Suggests using async/await
// Line 20: Suggests using callbacks
// Line 30: Suggests using promises
// Line 40: Back to callbacks

// PICK A LANE

Making AI Pair Programming Actually Work

1. Set Clear Context

// Start files with clear comments
/**
 * Frontend React component
 * Uses: TypeScript, MUI, React Query
 * No jQuery, No class components, No Redux
 */

2. Write Types/Interfaces First

interface User {
  id: string;
  email: string;
  role: 'admin' | 'user';
}

// Now AI knows what it's working with

3. Use AI as First Draft, Not Final Code

  • Let AI write the structure
  • You fix the logic
  • AI helps with tedious parts
  • You ensure it makes sense

The Workflow That Actually Works

1. Human: Define the function signature
2. AI: Suggests implementation
3. Human: "No, we're using PostgreSQL not MySQL'
4. AI: Updates to PostgreSQL
5. Human: "Add error handling"
6. AI: Adds try-catch
7. Human: "Use our custom error class"
8. AI: Still uses generic Error
9. Human: Manually fixes it
10. Both: Pretend this was productive

Real Productivity Metrics

What they claim:

  • "10x productivity boost!"
  • "Write code at the speed of thought!"
  • \"Never write boilerplate again!"

What I measured:

  • Boilerplate: 3x faster ✅
  • Complex logic: 0.8x (slower due to debugging)
  • Tests: 2x faster ✅
  • Refactoring: 1.5x faster ✅
  • Overall: ~1.4x productivity

Good? Yes. 10x? LOL no.

The Mental Model That Works

Stop thinking of AI as a "pair programmer." It's more like:

  • A very fast intern - Eager but needs supervision
  • Stack Overflow as a service - But sometimes outdated
  • Autocomplete on steroids - Helpful but not always right
  • A rubber duck that talks back - And sometimes talks nonsense

Rules for Sane AI Pairing

  1. Never trust, always verify - Especially imports and API calls
  2. AI writes structure, you write logic - Play to each other's strengths
  3. Comment your intentions - Helps AI stay on track
  4. Know when to turn it off - Complex algorithms? Do it yourself
  5. Use it for exploration - "How would you approach this?"

The Features I Actually Want

  • "This API doesn't exist" warnings
  • "You're in a browser environment" reminders
  • "This pattern is deprecated" alerts
  • "I'm not confident about this" honesty
  • Consistency in style within the same file

The Bottom Line

AI pair programming is like having a brilliant but absent-minded colleague. Fantastic for:

  • Getting unstuck
  • Writing tedious code
  • Exploring approaches
  • Learning new patterns

Terrible for:

  • Critical business logic
  • Performance-sensitive code
  • Security implementations
  • Anything requiring deep context

Use AI as a power tool, not a replacement for thinking. It's pair programming where you're always the senior developer, no matter what the AI claims to know. And always, always check if that npm package actually exists before trying to install it.

Get Started