```markdown --- title: "AI-Generated Code: Quality, Risk, and Review" subtitle: "What Changes When AI Writes the Code and Humans Review It" author: "David Kelly Price" version: "1.0" date: 2026-04-20 status: draft type: ebook target_audience: "Engineering managers, senior engineers, and tech leads managing teams where AI writes significant portions of code — concerned with quality, risk, and process" estimated_pages: 75 chapters: - "What AI-Generated Code Actually Looks Like" - "Where AI Gets Code Right and Where It Doesn't" - "The Shifting Role of Code Review" - "Security Risks Specific to AI-Generated Code" - "Testing What You Didn't Write" - "Dependency and License Risk" - "Building a Review Process for AI Code" - "Metrics: Measuring Code Quality When AI Is Involved" - "Team Culture and Skill Development" tags: - pyckle - ebook - ai-generated-code - code-review - quality - risk - engineering-management - draft --- --- # AI-Generated Code: Quality, Risk, and Review ## What Changes When AI Writes the Code and Humans Review It **By David Kelly Price** Version 1.0 — April 2026 --- ## Table of Contents - About This Guide - Chapter 1: What AI-Generated Code Actually Looks Like - Chapter 2: Where AI Gets Code Right and Where It Doesn't - Chapter 3: The Shifting Role of Code Review - Chapter 4: Security Risks Specific to AI-Generated Code - Chapter 5: Testing What You Didn't Write - Chapter 6: Dependency and License Risk - Chapter 7: Building a Review Process for AI Code - Chapter 8: Metrics: Measuring Code Quality When AI Is Involved - Chapter 9: Team Culture and Skill Development - Conclusion - Appendix A: Glossary - Appendix B: Tools & Resources - Appendix C: Further Reading --- ## About This Guide This book was written for engineering teams that have crossed a threshold. Not the threshold of experimenting with AI code generation — most teams are past that. The threshold of AI writing a meaningful share of production code, and humans being responsible for the outcome. That situation creates a set of problems that don't fit neatly into existing frameworks. The old code review process was built around humans reviewing other humans' code. The old testing philosophy assumed the person writing the tests understood the intent of the code. The old mental model of "who wrote this and why" collapses when the author is a language model that has no intent, no context about your system, and no stake in whether the code works next quarter. Teams that haven't updated their processes for this new reality are carrying risk they can't see. They're shipping AI-generated code with the same review standards they applied to code written by a senior engineer who understood the business logic. That's a mismatch. This guide addresses that mismatch head-on. It covers what AI-generated code actually looks like in practice, where models consistently fail, how review processes need to adapt, what security risks are specific to this context, and how to build team culture that doesn't erode engineering judgment while still capturing the productivity gains. The perspective here comes from working on AI/ML tooling and retrieval systems — from the inside of the problem, not from the sidelines. The goal isn't to convince you that AI coding tools are good or bad. They're tools. The goal is to help you use them well and manage the risks they introduce. --- ## Chapter 1: What AI-Generated Code Actually Looks Like Before building a review process, it helps to understand what you're reviewing. AI-generated code has identifiable characteristics — patterns that show up repeatedly, not because of any single model's quirk, but because of how these systems work. ### Surface Characteristics The most immediately obvious feature of AI-generated code is syntactic fluency. It looks correct. Variable names are sensible. Comments are present. The structure follows conventions. This is both the reason teams adopt AI tools and the reason they create new risks — code that looks right can be wrong in ways that aren't visible at first glance. Contrast this with code written by a junior engineer who's still building fluency. Badly structured, inconsistently named, logic that wanders. That code signals its own problems. AI-generated code doesn't. The surface is smooth. The problems are underneath. The second characteristic is completeness without coherence. When you ask an AI model to write a function, it writes a function that is complete — all branches handled, no obvious stubs. What it may not do is integrate coherently with the rest of the codebase. The function exists, but it doesn't know that your team standardized error handling three releases ago, or that the database abstraction layer you use has specific conventions for transactions. The model doesn't know your codebase unless you've given it that context explicitly. Third: AI code tends toward verbosity in some areas and compression in others, in ways that don't always map to appropriate abstraction. A model might write a fifty-line function that should be three separate concerns, because the prompt asked for one thing and it delivered one monolithic response. Or it might compress something into a one-liner that's technically correct but incomprehensible to the next person reading it. ### The Confidence Problem AI models generate text with no uncertainty markers unless specifically prompted to include them. When a model writes an implementation, it doesn't flag the three places where it made an assumption that might be wrong. It doesn't say "I'm not sure if this is the correct locking strategy for your use case." It produces complete, confident code. This is a fundamental mismatch with how experienced engineers write code. A good senior engineer leaves uncertainty visible — through comments, through tests that document edge cases, through PR descriptions that explain tradeoffs. A model leaves uncertainty invisible. The code looks like the product of someone who knew what they were doing. ```python # AI-generated function — looks correct, contains a quiet assumption def calculate_user_discount(user_id: str, order_total: float) -> float: user = db.get_user(user_id) if user.subscription_tier == "premium": return order_total * 0.15 elif user.subscription_tier == "standard": return order_total * 0.05 return 0.0 ``` This code is syntactically clean, handles multiple cases, and returns a sensible default. It also assumes that `db.get_user()` never returns `None`, that the subscription tier values are exactly those strings (case-sensitive), and that the discount percentages are business logic that hasn't changed since whatever training data the model saw. A reviewer who hasn't internalized the confidence problem might approve this in thirty seconds. ### Context Blindness AI code generation tools vary significantly in how much codebase context they ingest. Some have small effective context windows. Some index the codebase but do it shallowly. Some rely entirely on the current file and adjacent imports. Even the best current tools have incomplete understanding of the system they're generating code for. This produces a specific pattern: code that is correct in isolation but wrong in context. The function does what it says. It's just not what the codebase needs, or it duplicates something that already exists, or it makes an assumption that's valid in 90% of cases and catastrophic in the 10%. ```python # The model didn't know this already existed def get_active_users(db_session): return db_session.query(User).filter(User.is_active == True).all() # ...while elsewhere in models/user.py: class UserRepository: def get_active(self, session) -> List[User]: return session.query(User).filter( User.is_active == True, User.deleted_at.is_(None) # Soft delete — model didn't know ).all() ``` The AI wrote a function that looks correct. But the codebase already has a canonical query that includes a soft-delete check. Now there are two ways to get active users, one of which silently returns deleted accounts. ### The Training Data Artifact AI models are trained on code that was written at various points in time, of varying quality, from repositories with varying standards. This means they can and do generate patterns that were once common but are now considered bad practice. Deprecated APIs. Security patterns that were acceptable five years ago. Framework idioms from old versions. ```javascript // This was standard Express.js error handling — circa 2018 app.use(function(err, req, res, next) { console.error(err.stack); res.status(500).send('Something broke!'); }); ``` A model might generate this and it'll work. But your current error handling framework has observability, structured logging, error IDs, and alerting hooks. The model didn't know that. > **Key Insight** > AI-generated code is not authored in the way human code is authored. There's no intent, no mental model of the system, no awareness of accumulated technical decisions. The code is a statistical output shaped by training data. Treating it as something authored by a knowledgeable colleague is the single most common source of review failures. ### Volume and Velocity The last characteristic to understand is scale. AI tools dramatically increase the volume of code a single developer can produce. This is the point — that's why teams adopt them. But volume and velocity create their own risk profile. When a developer writes code by hand, there's a natural rate limit. That rate limit also applies to the number of subtle mistakes introduced per day. AI tools remove the rate limit on code volume but not on the human cognitive bandwidth available to review it. The result is a growing mismatch: more code, same or less review capacity per line. Teams that haven't accounted for this are in a situation where review quality per line of code is declining even as overall code volume increases. This isn't a hypothetical. It's observable in the review metrics of most teams that have adopted AI tools without updating their review process. > **Warning** > Velocity is not the same as productivity. Shipping more code faster is only better if the quality and correctness hold. Without updated processes, AI tools can increase technical debt faster than human tools ever could. ### Key Takeaways 1. AI-generated code is syntactically fluent and can look correct while being wrong in context-specific ways. 2. Models generate confident, complete code with no visible uncertainty markers — reviewers must compensate. 3. Context blindness means code correct in isolation may be wrong in your system. 4. Training data artifacts mean AI tools can introduce outdated patterns and deprecated APIs. 5. Volume increases without corresponding review capacity creates compounding risk. > **Try This** > Pull ten recent PRs where AI tools were used. Without running the code, read each diff as if you were reviewing it from scratch. Count how many times you caught yourself thinking "this looks fine" and moving on without checking whether it integrates correctly with surrounding code. That number tells you something about where your review process needs work. --- ## Chapter 2: Where AI Gets Code Right and Where It Doesn't Not all code is equally hard to generate correctly. AI models have consistent strengths and consistent failure modes, and understanding both is necessary for building a review process that allocates scrutiny correctly. ### Where AI Is Genuinely Reliable **Boilerplate and structural code.** When the task is scaffolding — setting up a class hierarchy, creating data models from a spec, implementing a standard CRUD pattern — AI tools are excellent. The patterns are well-represented in training data, the rules are unambiguous, and there's limited room for context-specific failure. A model asked to generate SQLAlchemy models from a database schema will almost always produce correct, idiomatic code. **Algorithmic implementations with clear specs.** When a problem is well-specified and the solution is unambiguous — sorting algorithms, string manipulation, mathematical calculations — AI tools perform reliably. The model has seen thousands of implementations of merge sort. It knows what the spec requires. The output is predictable. **Standard library usage.** When the task involves using standard libraries correctly — file I/O, JSON parsing, date manipulation, HTTP clients — models are generally reliable. These APIs are thoroughly documented and heavily represented in training data. ```python # AI is reliable here import json from datetime import datetime, timezone from pathlib import Path def read_config(config_path: str) -> dict: path = Path(config_path) if not path.exists(): raise FileNotFoundError(f"Config not found: {config_path}") with open(path, 'r', encoding='utf-8') as f: return json.load(f) ``` **Test generation for existing code.** Given a function with clear inputs and outputs, AI tools generate unit tests well. The test cases cover obvious paths, edge cases are handled. This doesn't mean the tests are correct — the model might test the wrong behavior — but the mechanical task of writing test structure is reliably executed. **Documentation and type annotations.** Writing docstrings, adding type hints to existing code, generating API documentation from function signatures. Models are consistently strong here because the task is essentially summarization and structural transformation of information that's already present. ### Where AI Fails Consistently **Business logic.** Any code that encodes domain rules that aren't universally understood fails to generalize from a model's perspective. The model doesn't know that your "order completion" logic has a special case for enterprise customers, or that the pricing model changed last quarter, or that certain states are only reached through a specific API path. It will write code that looks correct and implements a plausible interpretation of the business rule. Whether that's the right interpretation depends entirely on how much context was in the prompt. ```python # AI's version — plausible, wrong for this business def calculate_shipping(order: Order) -> Decimal: if order.total > 100: return Decimal('0') # Free shipping above $100 return Decimal('9.99') # Actual business rule — enterprise accounts always get free shipping, # subscription customers get free shipping above $50, standard above $100 # None of this was in the model's context ``` **State management and concurrency.** Code that involves shared mutable state, race conditions, and concurrent access is where models fail most dangerously. Not because they produce obviously wrong code, but because the bugs are subtle. Locks in the wrong place. Double-checked locking patterns that don't work in the language's memory model. Missing atomicity in multi-step operations. ```python # Looks right, has a race condition class ConnectionPool: def __init__(self, max_size: int): self.connections = [] self.max_size = max_size def get_connection(self): if len(self.connections) < self.max_size: conn = create_connection() self.connections.append(conn) return conn raise PoolExhaustedException() # Missing: thread safety — two threads can both pass the length check # and both create connections, exceeding max_size ``` **Error handling that's actually useful.** AI tools write try/except blocks reliably. They write error handling that catches errors and logs them or raises them. What they don't write well is error handling that's appropriate for the system — errors that carry useful context, errors that trigger the right alerts, errors that distinguish recoverable from unrecoverable failures in the way your architecture requires. **Integration with system-specific abstractions.** Whenever code needs to use an internal framework, custom ORM, internal service client, or any abstraction that isn't in the training data, model output degrades. The model will use the abstraction as if it were the standard library it most resembles. If your internal service client has different retry semantics than `requests`, the model doesn't know that. **Long-range dependencies within a file or system.** When the correctness of code depends on understanding relationships that span more than a few hundred lines — or worse, span multiple files — models fail. A function that sets up state that another function relies on. A context manager that assumes specific invariants about the calling code. The model sees the local code and writes locally correct code that violates distant invariants. > **Warning** > Concurrency and state bugs from AI-generated code are the highest-severity failure mode. They're hard to catch in review, they often don't surface in tests, and they manifest in production under load. Every piece of AI-generated code touching shared state deserves manual, detailed review. ### The Consistency Problem One subtle failure mode deserves its own treatment: AI models are inconsistent across runs. Ask the same model to implement the same function twice and you'll get different implementations. Both might be correct. But if your team is generating code with different context, on different days, across different prompts, the result is a codebase with multiple inconsistent solutions to the same problem. This is different from the inconsistency you get on a human team — at least human engineers discussing solutions converge toward agreement through code review. AI tools produce inconsistency silently, and without a process to catch it, it accumulates. > **Key Insight** > The AI's failure modes are not random. They cluster in predictable areas: business logic, concurrency, system-specific integrations, and long-range dependencies. A review process that treats all AI-generated code uniformly misallocates scrutiny. Invest more review time where models consistently fail. ### Calibrating Confidence by Task Type A practical framework for calibration: **High confidence, lighter review needed:** - CRUD operations on simple data models - Standard data transformations (CSV parsing, JSON transformation) - Algorithmic implementations from well-defined specs - Utility functions with no external state - Type annotations and documentation **Medium confidence, normal review:** - API endpoint implementation with clear contracts - Database query generation for known schemas - Standard library orchestration - Unit test generation for well-defined functions **Low confidence, heavy review required:** - Business logic with domain-specific rules - Anything involving shared state or concurrency - Code that integrates with internal systems - Error handling and recovery logic - Authentication and authorization - Any code that runs in production before being tested under realistic load ### Key Takeaways 1. AI reliably handles structural code, well-specified algorithms, standard library usage, and documentation. 2. Business logic, concurrency, and system-specific integrations are consistent failure areas. 3. AI error handling is syntactically correct but semantically shallow — it won't match your system's requirements. 4. Inconsistency across runs produces codebase drift that accumulates without a process to detect it. 5. Review effort should scale with task type, not with line count. > **Try This** > Take one week of AI-generated code from your team's PRs and categorize each function by task type: structural/boilerplate, algorithmic, business logic, concurrency, integration. Then look at which categories had bugs caught in review or post-merge. The distribution of bugs across categories will almost certainly match the failure patterns described above. --- ## Chapter 3: The Shifting Role of Code Review Code review was designed for a specific context: one engineer writes code, another engineer checks it. The reviewer's job was to catch bugs, enforce style, verify correctness, and share knowledge. Those goals haven't changed. But the nature of the code being reviewed has, and the review process hasn't kept up. ### What Review Was For Traditional code review serves several functions simultaneously. Bug detection is the obvious one, but in practice it's not the most valuable. Studies have consistently found that code review catches a modest percentage of bugs — automated testing and static analysis catch more. The deeper value of code review is knowledge transfer, architectural consistency, and accumulated team judgment. When a senior engineer reviews a junior engineer's code, the review conveys information in both directions. The reviewer learns what the junior engineer is building. The junior engineer learns how the senior engineer thinks about the problem. The review is a conversation. The code is the medium for the conversation. When one engineer reviews another's implementation choices, the result is a codebase where architectural decisions are deliberated, not just accumulated. Someone who sees a questionable abstraction can ask why it was made that way. Someone who sees a pattern that duplicates existing functionality can redirect it. The review creates shared understanding of the codebase. None of this works the same way when AI generates the code. ### What Changes First: knowledge transfer breaks down. The developer who submitted AI-generated code may not understand it well enough to defend it. The reviewer who approved it may have only checked that it looked plausible. Nobody has developed a mental model of how it works. The code exists in the repository without any human having fully internalized it. This is the most insidious long-term effect of AI code generation without updated review processes. Codebases accumulate code that nobody owns mentally. When it breaks, nobody knows why it was written that way. When it needs to change, nobody knows what it depends on. The codebase becomes opaque to the team that built it. Second: the reviewer's job shifts from judging whether the code is correct to judging whether the code should exist. An AI model asked to implement a feature will produce an implementation. The question is whether that implementation fits the codebase, serves the actual requirement, and doesn't duplicate or conflict with existing functionality. That's a higher-level judgment, and it requires the reviewer to understand both the code and the system. Third: the surface area of subtle incorrectness expands. Human-written code fails in ways that are often consistent with how the author misunderstood the problem. An experienced reviewer who knows the author can often predict where the bugs will be. AI-generated code fails in ways that are structurally unpredictable — the code looks competent, the bugs don't signal themselves, and the failure modes span the full range described in Chapter 2. > **Key Insight** > Review of AI-generated code requires more contextual knowledge, not less. The reviewer needs to understand the system well enough to evaluate whether the code fits, because the AI certainly doesn't. This is the opposite of the direction most teams are moving, where review is treated as a lower-effort task when AI is involved. ### The Accountability Gap Traditional code review creates accountability: an engineer writes code and takes responsibility for it. A reviewer approves it and takes shared responsibility. When something breaks, there are humans who can explain why decisions were made. AI-generated code creates an accountability gap. The engineer who submitted it may have barely read it. The reviewer may have treated it as lower-stakes because "the AI wrote it." When something breaks, the answer "the AI generated it and it looked right" is not a useful post-mortem finding. Teams need to explicitly close this accountability gap. The engineer who submits AI-generated code owns it fully — exactly as if they had written every line themselves. This has to be a stated norm, not an assumed one. If it's not stated, the natural tendency is to treat AI-generated code as lower-stakes, because psychologically it's easier to disclaim authorship of something you didn't write. ### What Reviewers Now Need to Do The mechanics of review need to adapt. The key additions: **Verify the requirement was understood.** Before reviewing the implementation, confirm the AI produced code that solves the actual problem. This sounds obvious but is consistently skipped. AI tools generate confident, complete implementations of a plausible interpretation of the prompt. That interpretation may be off. The first question in review is not "is this correct?" but "is this the right thing?" **Check for context blindness.** Does the code account for how the rest of the system works? Does it use existing abstractions? Does it duplicate functionality? Does it make assumptions about system state that aren't guaranteed? This check requires knowing the codebase, which means reviewers of AI-generated code need to be people with codebase context, not just people who can read code. **Evaluate the unhappy paths.** AI tools write happy paths confidently. Error handling, edge cases, and unusual inputs are where they fail more often. Force attention to these explicitly. **Trace state changes.** For any code that modifies state — especially shared state — trace what happens to that state before and after. Don't just read the function; follow the state through the system. **Run it.** Not just automated tests. Actually exercise the code against realistic inputs. AI-generated code can fail in ways that tests miss because the tests were also AI-generated against the same incorrect assumptions. > **Warning** > Approving AI-generated code you haven't fully understood is not a time-saving measure. It's deferred cost. The cost appears later, at higher interest, in production incidents and maintenance burden. ### Review as System Understanding The most valuable reframe for AI-code review is this: code review is now primarily a system understanding exercise, not a correctness checking exercise. The AI can produce code that passes all automated checks and still be wrong for your system. Only a reviewer who understands the system can catch that. This has implications for who should be reviewing AI-generated code. Junior engineers reviewing AI output are not well-positioned to catch system-level failures. They lack the context. This doesn't mean junior engineers shouldn't review code — the review is still a learning mechanism. But AI-generated code should route to reviewers with system context, not just to whoever is available. > **Try This** > In your next sprint, require that every PR containing significant AI-generated code include a brief explanation from the author: what they prompted the AI to do, what they verified about the output, and what they changed from the initial output. Don't make it bureaucratic — two or three sentences. Then review those explanations. They'll tell you immediately which engineers are actually owning the AI output and which are submitting it with minimal understanding. ### The Review Conversation One thing worth preserving is the conversational quality of review. Good code review generates dialogue: "why did you do it this way?", "have you considered this edge case?", "this duplicates what's in X". That dialogue builds shared understanding. With AI code, that conversation is harder because the author may not know why the AI did it a particular way. The productive version of the conversation shifts: "do you understand this code well enough to explain it?", "can you trace what happens if the database call fails here?", "does this match how we handle similar operations elsewhere?" If an author can't answer those questions, the code isn't ready to merge. That's not a criticism of using AI tools — it's a quality bar that protects the team. ### Key Takeaways 1. Traditional code review served knowledge transfer and architectural consistency, not just bug detection. AI code generation breaks both. 2. The reviewer's job shifts from correctness checking to evaluating fit — does this code belong in this system? 3. The accountability gap needs to be explicitly closed: submitting engineer owns AI-generated code fully. 4. Reviewers need system context, not just code reading ability. 5. Preserving the conversational quality of review prevents teams from accumulating code nobody understands. --- ## Chapter 4: Security Risks Specific to AI-Generated Code Security vulnerabilities are present in all code. AI-generated code introduces specific risk patterns that differ from what teams have traditionally managed. The difference isn't that AI is uniquely insecure — it's that the failure modes are different, and standard defenses may not catch them. ### The Training Data Problem AI models learn from code that existed when they were trained. That code includes security-vulnerable code, deprecated security patterns, and implementations that were acceptable before specific CVEs were discovered. Models don't distinguish between "this is the secure pattern" and "this was widely used before people understood the risk." The classic example is SQL injection. Modern models know not to concatenate SQL strings. But at a subtler level — parameterization in ORMs with raw query fragments, second-order injection through stored procedures, injection through JSON column queries — models revert to patterns that look clean but aren't. ```python # Model-generated — looks fine, isn't from sqlalchemy import text def get_user_orders(session, user_id: str, status_filter: str): # status_filter is used directly in the query query = text(f"SELECT * FROM orders WHERE user_id = :uid AND status = '{status_filter}'") return session.execute(query, {"uid": user_id}).fetchall() ``` The `user_id` is properly parameterized. The `status_filter` is string-formatted in. The model wrote half a secure query. This passes a quick read because the parameterization pattern is present. ### Overly Permissive Defaults AI tools, when in doubt, choose defaults that make things work. From a security standpoint, making things work often means being more permissive than necessary. CORS policies that accept all origins. Middleware that skips validation for internal networks. Authentication checks with overly broad exceptions. File upload handlers that accept any file type. These aren't malicious. They're the consequence of a model optimizing for a working implementation, not a secure one. ```javascript // AI-generated Express CORS — functional, insecure const cors = require('cors'); app.use(cors()); // Allows all origins // What it should be app.use(cors({ origin: process.env.ALLOWED_ORIGINS?.split(',') || [], methods: ['GET', 'POST', 'PUT', 'DELETE'], allowedHeaders: ['Content-Type', 'Authorization'] })); ``` The model wrote the simplest CORS configuration that works. It works. It also allows any website to make requests to your API. ### Insecure Cryptography Cryptographic code is an area where models are particularly dangerous. The pattern is: model-generated crypto code looks correct and uses real library functions. The implementation choices — key sizes, algorithms, modes of operation, IV handling — may be subtly wrong. ```python # AI-generated AES encryption — uses real library, wrong mode from Crypto.Cipher import AES import os def encrypt(data: bytes, key: bytes) -> bytes: cipher = AES.new(key, AES.MODE_ECB) # ECB mode — deterministic, unsafe return cipher.encrypt(data) ``` `AES.MODE_ECB` is in the library, is valid Python, and encrypts data. It's also not semantically secure for most use cases — identical blocks encrypt identically, leaking structure. A model will use it because it's simple and it works. > **Warning** > Never let AI-generated cryptographic code reach production without review by someone who specifically knows cryptographic best practices. The surface of "looks right, isn't right" is enormous in this domain. ### Dependency and Injection AI tools recommend dependencies based on training data. They recommend well-known libraries, but they don't always recommend the latest secure version. More importantly, they may recommend libraries that have been abandoned, superseded, or that have known vulnerabilities not yet in the training data. There's a subtler injection problem too: prompt injection. If your AI coding assistant is operating in an agentic mode — reading files, executing commands, accessing external systems — carefully crafted malicious content in those files or systems can influence the model's outputs. An attacker who knows your team uses an AI coding assistant could attempt to inject instructions into codebases, documentation, or API responses that the assistant processes. This is a new threat model that most teams haven't considered. If your AI tooling reads external data sources as part of code generation, those sources are attack surface. ### Authentication and Authorization The most dangerous category of AI security failures is authentication and authorization logic. Not because models write obviously broken auth — they don't. Because models write auth that's correct in the happy path and incorrect in edge cases. Missing permission checks on indirect object references. Authorization checks that are present but applied at the wrong layer. Role checks that don't account for role inheritance. Session validation that's correct for the primary path but missing for API key authentication. ```python # AI-generated endpoint — auth decorator present, but incomplete from functools import wraps def require_auth(f): @wraps(f) def decorated(*args, **kwargs): token = request.headers.get('Authorization') if not token: return jsonify({'error': 'Unauthorized'}), 401 # Missing: token validation, expiry check, user permission verification return f(*args, **kwargs) return decorated @app.route('/api/admin/users') @require_auth def list_users(): # This checks that a token exists, not that it's valid or has admin scope return jsonify(User.query.all()) ``` The decorator is there. The route is decorated. The auth check passes. But the decorator only checks for the presence of a header, not the validity of the token or the permissions of the user. This passes code review easily because the pattern is correct. > **Key Insight** > Security failures in AI-generated code cluster at the boundary between patterns and correctness. The model knows the pattern — auth decorator, parameterized query, CORS configuration. It may not implement the pattern correctly for your security requirements. ### Secrets Handling Models generate code that works. Working code often needs credentials. Models, when generating working examples or templates, sometimes include credential patterns that shouldn't make it to production — hardcoded connection strings, placeholder secrets that get committed, environment variable patterns that don't actually keep secrets out of code. ```python # AI-generated database connection — seen in production codebases DATABASE_URL = os.getenv('DATABASE_URL', 'postgresql://admin:password@localhost/mydb') # The default value is a real connection string with credentials. # It won't be used in production, but it signals a pattern that # developers may replicate with real credentials. ``` This specific code probably won't cause a breach. But it normalizes a pattern where credentials appear in source files, and that normalization leads to the breach. ### Building Security Into the Review Process The practical response to AI-specific security risks requires adding specific checks to review: **Automated checks first.** Run SAST tools (static analysis) on all AI-generated code before human review. SAST catches the mechanical patterns — SQL injection, XSS, insecure crypto primitives. Automating these checks removes the burden from reviewers and catches what's catchable automatically. **Manual security review for high-risk areas.** Authentication, authorization, cryptography, and any code that processes external input should require manual security review from someone with specific security knowledge, not just a passing read. **Secret scanning.** Every PR should run through secret scanning tools. AI-generated code has above-average rate of credential-pattern inclusion because it's generating functional examples. **Threat modeling for new endpoints.** Any new API surface — particularly AI-generated endpoint scaffolding — needs threat modeling. Who can reach this? What can they do? What's the worst-case input? ### Key Takeaways 1. AI models learn from historically vulnerable code and replicate patterns that are outdated or insecure. 2. Overly permissive defaults are a consistent failure mode — AI optimizes for working, not for minimal permissions. 3. Cryptographic code from AI tools requires specialized review — the library calls look correct while the implementation choices may not be. 4. Auth and authz are the highest-risk categories — correct pattern, incorrect implementation is common. 5. SAST automation handles mechanical checks; manual security review handles semantic correctness. > **Try This** > Run a SAST tool against the last three months of merged AI-generated code. Don't fix anything yet — just inventory the findings. Categorize them by type (injection, auth, crypto, defaults, etc.). The distribution will tell you where your team's AI tooling is most consistently producing security risk, which tells you where to focus review effort and possibly where to add automated gates. --- ## Chapter 5: Testing What You Didn't Write Testing AI-generated code requires the same thinking adjustment as reviewing it: the person writing the tests needs to understand the code, not just generate tests against it. And when AI tools generate both the code and the tests, a specific kind of blind spot appears that traditional testing philosophy doesn't address. ### The Aligned Failure Problem When an AI model writes a function and then writes tests for that function, both are generated from the same understanding of the problem. If that understanding is wrong, both the code and the tests will be wrong in the same direction. The tests will pass. The code will be wrong. ```python # AI-generated function — has an off-by-one error def paginate(items: list, page: int, page_size: int) -> list: start = page * page_size # Should be (page - 1) * page_size if pages are 1-indexed return items[start:start + page_size] # AI-generated tests — all pass, all test the wrong behavior def test_paginate_first_page(): items = list(range(100)) result = paginate(items, 0, 10) # Tests page 0, not page 1 assert result == list(range(10)) # Correct for 0-indexed, wrong for 1-indexed def test_paginate_second_page(): items = list(range(100)) result = paginate(items, 1, 10) assert result == list(range(10, 20)) # Passes, but this is "page 2" not "page 1" ``` All tests pass. The function works consistently. The question of whether pages should be 0-indexed or 1-indexed was never surfaced because both the code and the tests were generated with the same (unstated) assumption. This is the aligned failure problem: the tests confirm the code, but the code confirms itself. The test suite produces confidence without correctness. ### Human-Written Tests for AI-Generated Code The most robust mitigation is straightforward: require human-written tests for AI-generated logic. The person writing the tests has to understand what the code is supposed to do well enough to specify expected behaviors without asking the AI. This sounds laborious. It isn't, when framed correctly. The human-written tests don't have to cover everything. They specifically need to cover: 1. The specification of the function: what does it actually mean for this to be correct, per the business requirement? 2. Edge cases that require domain knowledge: what inputs are meaningful in this system that a generic test wouldn't consider? 3. Integration behavior: how does this code interact with the surrounding system, and are those interactions tested? The mechanical test cases — null inputs, empty collections, type checking — can be AI-generated. The specification-level tests must be human-written, because they encode human understanding of the requirement. > **Key Insight** > Test generation by AI is most valuable for mechanical coverage — boundary values, type variations, exception paths. It's least valuable for specification testing, where the question is whether the code does the right thing, not whether it does the thing consistently. ### Property-Based Testing as a Complement Property-based testing is particularly valuable for AI-generated code because it separates the specification of properties from the generation of test cases. A human specifies what should always be true. A testing framework generates the cases. ```python from hypothesis import given, strategies as st # Human-written: specifies what should always be true @given( items=st.lists(st.integers()), page=st.integers(min_value=1, max_value=100), page_size=st.integers(min_value=1, max_value=50) ) def test_paginate_never_returns_more_than_page_size(items, page, page_size): result = paginate(items, page, page_size) assert len(result) <= page_size @given( items=st.lists(st.integers(), min_size=1), page=st.integers(min_value=1), page_size=st.integers(min_value=1, max_value=50) ) def test_paginate_results_are_subsets_of_input(items, page, page_size): result = paginate(items, page, page_size) assert all(item in items for item in result) ``` Property-based tests written by humans against AI-generated code provide strong coverage without requiring human enumeration of every test case. The properties are specification; the cases are generated. ### Mutation Testing Mutation testing — deliberately introducing small bugs into code and checking whether tests catch them — is a powerful tool for evaluating test quality. It answers the question: are these tests actually checking anything? For AI-generated code where tests were also AI-generated, mutation testing frequently reveals that the tests are insufficiently specific. They pass for the original code, but they also pass for mutated versions, meaning they're not actually validating the intended behavior. Tools like `mutmut` (Python), `PIT` (Java), or `Stryker` (JavaScript/TypeScript) can run against a codebase and produce mutation scores. A low mutation score for AI-generated code is a strong signal that the test suite was generated mechanically rather than written with specification intent. ### Integration Testing Requirements Unit tests for AI-generated code address function-level correctness. Integration failures — the places where AI code fails to fit the system — require integration tests. And integration tests for AI code need to be written with explicit attention to the failure modes from Chapter 2. Specifically: - Tests that exercise the code against a real database, not a mock, because AI database code often makes wrong assumptions about transaction behavior. - Tests that verify the code uses existing abstractions correctly, not just that it runs without error. - Tests that cover the authorization paths, not just the happy path. - Load tests for any concurrent code, because concurrency bugs in AI-generated code are common and don't appear under single-threaded test conditions. > **Warning** > An integration test suite that only tests the happy path against AI-generated code is not a safety net. It's a confidence generator. If the AI code has a concurrency bug or an incorrect assumption about database behavior, happy-path integration tests won't catch it. ### Test Coverage Is Not the Same as Test Correctness This is true generally but is particularly important for AI code. A model asked to improve test coverage will improve test coverage. It will write tests that exercise lines of code. Those tests may not actually verify correctness. ```python # This test achieves 100% line coverage on the function # It does not verify that the output is correct def test_process_payment(): result = process_payment(order_id="123", amount=50.00, method="card") assert result is not None # Passes for any non-None return, including error states ``` Coverage-driven test generation produces tests with no assertions, or assertions that are vacuously true. Coverage metrics can look excellent while providing no actual safety net. ### Regression Testing After AI Refactors When AI tools are used for refactoring — not just writing new code but modifying existing code — regression testing becomes critical. The specific risk is that a refactor that appears to preserve behavior actually changes it in edge cases the tests don't cover. Before any AI-assisted refactor, the test suite needs to be known-good. If the test suite wasn't comprehensive before the refactor, the refactor may have introduced regressions into areas that were already untested. Snapshot testing — capturing the output of functions against a set of known inputs before the refactor and comparing after — can catch behavioral changes that unit tests miss. It's imprecise (it captures bugs too, not just correct behavior), but it's a useful safety net for AI-driven refactors. > **Try This** > Pick three AI-generated functions from recent PRs where tests were also AI-generated. Write replacement tests from scratch, starting only from the business requirement, not from the code. Compare your tests to the AI-generated tests. Count how many cases the AI tests missed. Count how many cases your tests caught that the AI tests marked as correct. That delta tells you the risk in your current test-generation process. ### Key Takeaways 1. AI-generated tests for AI-generated code create aligned failure: both can be consistently wrong. 2. Human-written tests should cover specification-level correctness; AI-generated tests handle mechanical coverage. 3. Property-based testing separates specification (human) from test case generation (automated). 4. Mutation testing evaluates whether tests are actually checking anything. 5. Coverage metrics for AI-generated test suites are unreliable proxies for correctness. --- ## Chapter 6: Dependency and License Risk AI coding tools make decisions about which external libraries to use. Those decisions happen quickly, often without consideration of security, licensing, or maintenance status. The dependency choices embedded in AI-generated code represent a class of risk that most teams haven't explicitly addressed. ### How AI Chooses Dependencies Models recommend dependencies based on what was popular and commonly used in their training data. This creates several risk patterns: **Popularity bias.** A model recommends libraries that were widely used when the training data was assembled. Libraries that have been superseded by better alternatives may still be recommended because the older library had more code examples. Libraries that have since been abandoned may still appear as recommendations. **Version blindness.** Models generally don't track current package versions or security advisories. A recommended library version might have known CVEs published after the training cutoff. **Functional equivalence confusion.** The model knows that several libraries solve a given problem. It picks one based on statistical frequency in training data, not based on which best fits your existing dependency tree, your team's familiarity, or your licensing requirements. ### License Risk in AI-Generated Dependencies When a model recommends a dependency and your team installs it, your project has incorporated that library's license terms. If your project is commercial software, incompatible licenses create legal risk. This includes: - **GPL/LGPL contamination**: Including GPL-licensed code in a commercial closed-source product without understanding the implications. - **AGPL requirements**: AGPL libraries require source code disclosure when the software is used over a network, which affects SaaS products. - **Attribution requirements**: Many permissive licenses (MIT, Apache 2.0, BSD) require attribution in distributed software. Missing attribution in commercial releases creates legal exposure. The risk isn't hypothetical. Several notable companies have faced legal challenges over open-source license violations in commercial software, often discovered years after inclusion. AI tools don't evaluate license compatibility. They recommend what works. Your team needs a process to evaluate what's legally permissible. ```bash # Before merging any PR with new dependencies, run license checks pip-licenses --format=table --with-urls # or for Node: license-checker --summary ``` ### Dependency Chain Risk When an AI tool adds a dependency, it adds the dependency's dependencies as well. A seemingly simple "add a markdown parsing library" can pull in a transitive dependency tree of twenty packages. Any of those packages might have security vulnerabilities, restrictive licenses, or stability problems. ``` requests==2.31.0 ├── charset-normalizer>=2,<4 ├── idna<4,>=2.5 ├── certifi>=2017.4.17 └── urllib3<3,>=1.21.1 ``` This is a small, clean dependency tree. Many libraries are not this clean. An AI-generated implementation that imports five packages might introduce forty transitive dependencies. The surface area for supply chain attacks and license violations multiplies with each transitive dependency. > **Key Insight** > The dependency a model recommends is the entry point, not the entire risk. Every new dependency is also an adoption of its entire dependency tree. Audit the tree, not just the package. ### Dependency Confusion and Supply Chain Attacks Supply chain attacks through dependency confusion — where an attacker publishes a malicious package under a name that conflicts with an internal package name — are a real and demonstrated attack vector. AI tools that recommend packages don't distinguish between internal and external package names. If an engineer follows an AI recommendation and installs a package without checking the source, dependency confusion attacks become easier to execute. The practical mitigation: all AI-recommended package installations should go through the same verification process as any other dependency addition. Check the package source, check the maintainer, check the publication date, check whether it's the canonical package for this purpose. ### Evaluating Dependency Health Beyond security and licensing, dependency health matters for long-term maintenance. An AI tool might recommend a library that: - Has had no commits in three years - Has unresolved critical issues in its tracker - Is maintained by a single person with no organizational backing - Was the right tool in 2019 but has been superseded None of this is visible to the AI. It only knows that the library exists and was used. The engineering team has to evaluate health. A quick checklist for any AI-recommended dependency: - [ ] Last release date — more than two years without a release is a yellow flag - [ ] Maintenance status — is the project marked as maintained? - [ ] Download trends — declining downloads may signal community migration away from the library - [ ] Open security issues — check GitHub security advisories and CVE databases - [ ] License — verify compatibility with your project's license - [ ] Dependency tree — scan for deep or unstable transitive dependencies - [ ] Alternatives — is there a more standard library that handles this in your ecosystem? ### Automated Dependency Scanning Manual evaluation can't scale to every AI-generated dependency addition. Automated scanning should be a gate in the CI/CD pipeline for all PRs that modify dependency manifests. **For security:** - `npm audit`, `pip-audit`, or `Snyk` for vulnerability scanning - `dependabot` or `renovate` for automated update PRs on known-vulnerable versions - `osv-scanner` for checking against the Open Source Vulnerability database **For licensing:** - `fossa` for commercial-grade license compliance scanning - `license-checker` (Node) or `pip-licenses` (Python) for quick audits - `scancode-toolkit` for deeper license detection in pulled source **For supply chain:** - `sigstore/cosign` for artifact signing verification - Lock files committed to version control (package-lock.json, Pipfile.lock, etc.) - Private registries with allowlists for organizational control > **Warning** > Automated dependency scanning produces findings. Those findings need a response process. A scanner that flags vulnerabilities but generates no tickets and triggers no fixes is security theater. The scan is only the first step — there needs to be a human process that responds to findings. ### The Lock File Discipline AI tools, when generating project setup code or build configurations, don't always include lock files. Or they include incomplete lock files. Or they generate `requirements.txt` with ranges (`requests>=2.28`) instead of pinned versions. Unpinned dependencies are a supply chain risk vector. A range specifying `>=2.28` will resolve to whatever the latest version is at install time, which means a supply chain attack on any version in that range affects your build. All AI-generated dependency specifications should be reviewed for pinning discipline. Lock files should be committed. CI should run against the locked versions, not the ranges. ### Key Takeaways 1. AI recommends dependencies based on training data popularity, not current security posture, license compatibility, or maintenance health. 2. Every new dependency imports its entire transitive tree — audit the tree. 3. License compatibility must be verified manually; AI tools don't evaluate license terms against your project's requirements. 4. Automated dependency scanning (security and licensing) should be a CI gate, not a periodic manual task. 5. Unpinned dependencies are a supply chain risk; all AI-generated dependency specifications should use lock files. > **Try This** > Run a license audit against your current dependency tree using the tools listed above. Count the number of licenses present. Identify any that are potentially incompatible with your project's license or distribution model. Then trace which of those were added in PRs that contained AI-generated code. This exercise usually produces a few surprises about what your team has actually shipped. --- ## Chapter 7: Building a Review Process for AI Code The previous chapters describe a set of risks. This chapter is about what to actually do. A review process for AI-generated code isn't a separate system — it's a set of modifications to the existing process that account for the specific failure modes of AI-generated code. ### The Core Principle The core principle is simple, even if the implementation isn't: the review process needs to verify what the AI can't guarantee. The AI can guarantee syntactic correctness. It cannot guarantee semantic correctness for your system, appropriate security, correct business logic, or fit with your architecture. Every element of a review process for AI code should derive from that principle. If a check is catching something the AI reliably gets right, it's redundant. If a check is verifying something the AI cannot guarantee, it belongs. ### Triage First: Not All AI Code Is Equal Before adding review steps, acknowledge that AI-generated code varies in risk. A ten-line utility function for formatting dates doesn't need the same review process as a new authentication middleware. Start with triage. **Risk tiers:** *Low risk:* Pure functions with no external state, utility code with well-defined inputs and outputs, documentation and type annotations, structural/boilerplate code. *Medium risk:* API endpoint implementations, database query code, business logic with clear specs, test code. *High risk:* Authentication and authorization, cryptographic operations, concurrent or distributed state management, external integrations, any code that handles payment, health, or personally identifiable information. The triage determines the review tier. Low-risk code can follow a lighter process. High-risk code needs additional scrutiny regardless of who wrote it. ### The Review Checklist Rather than a vague mandate to "review more carefully," give reviewers a structured checklist for AI-generated code. Checklists are not bureaucracy for their own sake — they externalize cognitive load, prevent pattern blindness, and create accountability. A working checklist for AI-generated code review: **Before reading the code:** - [ ] What was the original requirement? Does the PR description state it clearly? - [ ] Is this in a high-risk tier (auth, concurrency, external integrations)? - [ ] Are there existing patterns in the codebase this should follow? **While reading the code:** - [ ] Does the code solve the stated requirement — or a plausible but incorrect interpretation of it? - [ ] Does it use existing abstractions and utilities, or does it duplicate them? - [ ] Are all external inputs validated? - [ ] Are error conditions handled appropriately for this system? - [ ] For any state modification: is the state management consistent with how the rest of the system handles state? - [ ] Are there implicit assumptions that aren't stated or enforced? **Security checks:** - [ ] Are all user-facing inputs sanitized before use? - [ ] Does any database query use parameterization? - [ ] Are permission checks present and semantically correct? - [ ] Are any secrets present in the code (even as defaults or examples)? - [ ] Does any new dependency have a known vulnerability or restrictive license? **Testing:** - [ ] Are the tests human-written for specification-level correctness? - [ ] Do the tests cover the business requirement, not just the implementation? - [ ] Are error paths tested? **After review:** - [ ] Can the submitting engineer explain any part of this code on demand? That last item is important. The understanding check — "can you explain this code if I ask?" — is the accountability gate. If the answer is no, the code isn't ready. > **Key Insight** > Checklists reduce cognitive load under time pressure. Reviewers who are reviewing a dozen PRs a day will find their thoroughness degrading under load. A checklist keeps the minimum bar consistent even when attention varies. ### Automating the Automatable Review effort should be reserved for what humans can catch and machines cannot. Everything that can be automated should be. **Gate 1 — Static analysis.** Run a SAST tool on every PR as part of CI. Common choices by ecosystem: `bandit` (Python security), `semgrep` (multi-language, rule-based), `sonarqube` (comprehensive, commercial), `ESLint` with security plugins (JavaScript). Gate merges on critical findings. **Gate 2 — Dependency scanning.** Run vulnerability and license scanning on any PR that modifies dependency manifests. Gate merges on critical vulnerabilities. Log and ticket high-severity findings. **Gate 3 — Secret scanning.** Run secret detection on every commit. `git-secrets`, `truffleHog`, `gitleaks`, or GitHub's built-in secret scanning. Never let a commit with credentials reach the main branch. **Gate 4 — Type checking.** For typed languages, run the type checker in strict mode as a CI gate. AI-generated code often has type errors that only surface with strict type checking enabled. **Gate 5 — Test quality.** Run mutation testing as a periodic gate (not every commit — it's slow). Flag PRs where AI-generated tests have mutation scores below a threshold. The goal of automated gates is to prevent reviewers from spending time on mechanical checks. A reviewer who has to find SQL injection in a PR is doing work a scanner should have done. ### The Author's Responsibility Review processes put the burden on the reviewer. For AI-generated code, the author needs to bear more of the burden than traditional code review assumes. **The author should document:** - What prompt or instruction produced the AI output - What they verified before submitting - What they changed from the initial output - Any areas where they're uncertain about the implementation This doesn't have to be extensive. Three to five sentences in the PR description. But it shifts the accountability in the right direction and gives reviewers the information they need to focus their attention. **The author should test locally.** Not just automated tests — actually exercise the feature with realistic inputs before submitting. If the author has found that the code works for real inputs, the reviewer is validating, not discovering. **The author should understand the code.** This is the non-negotiable. An engineer who submits code they can't explain has not done their job. AI tools are code generation assistants, not code substitutes. The engineer owns the output. > **Warning** > Building a review process that compensates for engineers who don't understand their own submissions will fail. The process can add checks, but it can't add understanding. If engineers are submitting AI-generated code they haven't internalized, address that cultural issue directly. The process can't fix it. ### Reviewer Pairing by Context One structural change worth making: route AI-generated PRs to reviewers with system context, not just to whoever's next in the rotation. The reviewer who can catch AI-generated code failures is the reviewer who knows the codebase — who will recognize that a function duplicates an existing abstraction, or that an API assumption is wrong, or that the error handling doesn't match the system's conventions. That's not just seniority. It's familiarity with the relevant part of the codebase. For large codebases with distinct domains, code ownership models (CODEOWNERS in GitHub) can enforce that PRs in specific areas route to engineers with context in those areas. This is good practice for all code and particularly important for AI-generated code. ### Staged Merge Requirements For high-risk AI-generated code, consider staged merge requirements: 1. **Automated gates pass** — CI completes, all automated checks green 2. **Reviewer with domain context approves** — not just any reviewer 3. **Author confirms understanding** — verbal or written attestation that they understand the code 4. **Post-merge monitoring period** — for production changes, active monitoring for N hours before considering the change stable Staged requirements create audit trails and force the team to be explicit about risk levels. A PR that goes through three explicit approvals is different from one that gets approved by whoever had time. ### Key Takeaways 1. Review effort should be concentrated on what AI cannot guarantee: semantic correctness, security, system fit. 2. Triage by risk tier before applying review intensity — not all AI code carries equal risk. 3. Structured checklists prevent pattern blindness and create accountability without being bureaucratic. 4. Automate the automatable: SAST, dependency scanning, secret scanning, type checking. 5. The author bears more responsibility with AI code, not less — documentation, local testing, and understanding are non-negotiable. > **Try This** > Draft a two-page review runbook for your team specific to AI-generated code. Include the triage criteria, the checklist, the automation gates, and the author documentation requirements. Share it with the team and run one sprint with it actively in use. Then collect feedback: where does the process add value and where does it create friction without benefit? Iterate from there. --- ## Chapter 8: Metrics: Measuring Code Quality When AI Is Involved Measuring software quality has always been imperfect. Lines of code tell you nothing meaningful. Test coverage tells you something but not enough. Cyclomatic complexity tells you about maintainability risk but not about whether the code is correct. Add AI to the picture and the existing metrics become even less reliable. ### What Gets Harder to Measure **Test coverage loses meaning.** When AI generates code and tests, coverage numbers can look strong while the test suite is nearly meaningless. A file with 95% line coverage but AI-generated tests that only check non-None returns is a false indicator. **Commit velocity distorts.** AI tools dramatically increase commit frequency and lines of code per commit. Traditional productivity metrics — PRs merged, lines written, commits per week — become misleading when AI is involved. A developer shipping ten PRs a week with AI tools is not necessarily more productive than a developer shipping three well-considered PRs. **Code review time becomes ambiguous.** Review time for AI-generated code that was approved quickly might mean the review was efficient, or it might mean the reviewer treated AI-generated code as lower-stakes. Without context, the metric is ambiguous. **Static analysis findings per engineer.** If AI-generated code has higher rates of static analysis findings, attributing those findings to the AI rather than the engineer matters for process improvement. Most tools don't distinguish the origin of code. ### Metrics That Do Signal Something Meaningful **Bug escape rate by code origin.** Track whether bugs that escape to production are disproportionately found in AI-generated code versus human-generated code. If they are, your review process isn't catching the AI failure modes. This requires tagging PRs or commits with origin information, which most teams don't do by default. **Review rework rate.** When a PR requires significant changes after initial review, that's a quality signal. Track how often PRs are returned for substantial rework versus minor nits. AI-generated code that sails through review but fails in QA has a specific pattern: low review rework rate, high QA failure rate. **Security finding rate in automated scans.** The rate of SAST findings in AI-generated code is a meaningful signal — not for judging the AI, but for calibrating where to focus review effort. If 80% of your SAST findings come from API endpoint code, that's where the review process needs improvement. **Post-merge defect density.** For a cohort of code changes, track the number of bugs found post-merge per unit of code changed. This is slower (requires waiting for bugs to appear) but captures what matters: defects in production. **Understanding verification rate.** If your review process includes author understanding checks, track how often they're failing. A high failure rate means engineers are submitting code they don't understand. That metric drives cultural change. > **Key Insight** > The most valuable metrics measure outcomes (defects, rework, security findings) rather than outputs (lines of code, PRs merged, coverage percentage). AI tools change the relationship between outputs and outcomes. Metrics that measure outputs become unreliable. Metrics that measure outcomes remain valid. ### Building Attribution into Your Workflow To measure AI-code quality separately from human-code quality, you need attribution. This requires minimal process overhead to be sustainable. **Option 1: PR labels.** Require engineers to label PRs with the origin of code. Labels like `ai-generated`, `ai-assisted`, `human-written`. Not judgment — just categorization. This makes it possible to segment metric data by origin. **Option 2: Commit convention.** Add a convention to commit messages for AI-generated code. GitHub Copilot, Cursor, and other tools sometimes add markers automatically. Standardize this. **Option 3: File-level tagging.** For codebases with clear modules, some teams tag modules or files that were primarily AI-generated in their documentation. This is coarser but works for larger-grained analysis. Attribution creates the ability to ask questions like: what's the post-merge defect rate for AI-generated code vs. human-written code? Which review checklist items are catching the most issues in AI code? These questions are impossible without attribution. ### Tracking Reviewer Effectiveness Reviewers of AI code need feedback on whether their reviews are effective. Without that feedback, review quality can't improve. **Review discovery rate.** Of the bugs found in a cohort of AI-generated code, what fraction were caught in review versus found in QA versus found in production? A declining review discovery rate (more bugs reaching QA and production) signals that review is becoming less effective. **Reviewer-specific discovery rate.** Which reviewers catch the most issues in AI-generated code? This information, used carefully, can identify who has the system context and review approach to be most effective — and can inform who should be reviewing what. This is sensitive data. It shouldn't be used punitively. But it is genuinely useful for process improvement. A reviewer who consistently misses issues in AI-generated code might benefit from different training, different pairing, or routing away from high-risk AI PRs. ### Quality Gates as Leading Indicators Outcome metrics (defects in production) are lagging — they tell you what happened after the fact. Quality gates in the CI/CD pipeline are leading indicators: they measure code properties that correlate with future defects. Track the gate failure rates over time: - What percentage of PRs fail SAST checks? Is that trending up or down? - What percentage of PRs fail secret scanning? Each failure is a near-miss. - What percentage of PRs fail dependency vulnerability scans? - What's the type distribution of gate failures? Are new failure types appearing? Gate failure trends tell you something about the code quality being produced. Rising failure rates signal either that code quality is degrading or that gates are becoming more stringent — and distinguishing those requires looking at the rule set. > **Warning** > Don't incentivize engineers to suppress or bypass automated quality gates to hit velocity metrics. A metric that creates pressure to merge code faster will produce code that fails gates more often — and engineers who find ways to silence the tools rather than fix the findings. ### The Velocity Trap One measurement trap deserves explicit treatment: using velocity as a proxy for AI-tool value. Teams that adopt AI tools often measure success by velocity increases: PRs per engineer per week, sprint velocity points, features shipped per quarter. These go up with AI tools. That's real. But velocity without quality tracking tells an incomplete story. If velocity goes up 40% and defect rate goes up 60%, the tools are net-negative. The faster shipping is creating more cleanup work than it's saving. This calculation requires tracking both numbers — which most teams don't do when adopting AI tools. A complete measurement framework tracks: - Velocity (output) - Defect rate (outcome) - Review rework rate (process quality) - Time to resolve production issues (operational quality) - Test coverage quality, not just coverage percentage Velocity without the other metrics is incomplete. The goal is to increase velocity without increasing defect density. That's achievable — but only if you're measuring both. ### Key Takeaways 1. Traditional metrics (coverage, lines of code, velocity) become unreliable proxies when AI is involved — outcomes matter more than outputs. 2. Bug escape rate by code origin is the most meaningful metric and requires attribution. 3. Attribution (PR labels, commit conventions) enables the segmentation needed to measure AI code quality separately. 4. Gate failure rate trends are leading indicators of code quality direction. 5. Velocity without defect tracking is an incomplete metric — measure both and hold both accountable. > **Try This** > Set up attribution for your next sprint: add a mandatory PR label for AI-generated, AI-assisted, and human-written code. At the end of the sprint, segment your quality metrics (SAST findings, review rework, QA failures) by attribution. The segmented data will either confirm or challenge your assumptions about where quality risk is concentrated. Either way, it's better information than you had before. --- ## Chapter 9: Team Culture and Skill Development The hardest part of managing AI-generated code isn't technical. The review checklists, the automated gates, the metrics — those are solvable with process. The hard part is culture: how a team relates to the code it produces, how skills develop or fail to develop, and what happens to engineering judgment when AI handles more of the mechanical work. ### The Deskilling Risk Every major productivity tool carries some deskilling risk. Calculators made mental arithmetic less practiced. IDEs made syntax memorization less necessary. AI code generation takes this further: it can make the act of writing code less practiced. For experienced engineers, this is mostly a non-issue. They have mental models built from years of practice. They use AI as a productivity tool without losing the underlying capability. But for engineers in the earlier stages of career development, AI tools used incautiously can interrupt the learning process. Learning to code involves building a mental model: understanding why things fail, developing intuition for edge cases, recognizing patterns across problems. That model is built through friction — writing code that doesn't work and figuring out why, reading others' code and understanding the decisions, debugging production issues that reveal system behavior. AI tools can short-circuit that friction. A junior engineer who gets working code from AI on every problem may be shipping features, but may also be failing to build the mental models that make a senior engineer effective. When the AI fails — and it will — that engineer doesn't have the model to diagnose why. > **Key Insight** > Velocity gains from AI tools are immediate and visible. Skill development impact is slow and invisible until it isn't. The engineer who never learned to debug because the AI fixed it every time will struggle when the AI generates something that requires real diagnosis. That lag makes the risk easy to miss. ### Intentional Learning Structures Protecting skill development in an AI-assisted team requires intentional structure. Some practices that work: **Require manual implementation of core concepts.** Designate certain problem categories — algorithms, system design components, critical business logic — where engineers write the first implementation by hand. Not forever, not for everything. But the practice of building something from scratch is how mental models form. Making it optional means it won't happen. **Code walkthroughs, not just code reviews.** Schedule periodic walkthroughs where engineers explain code they've written — or had AI write — in detail. Not a performance review; a learning exercise. The engineer who can explain the code has understood it. The one who can't reveals a gap. **Debug without AI.** When production issues arise, resist the temptation to immediately prompt the AI for a fix. Use it as a teaching opportunity: have the engineer diagnose the issue manually, form a hypothesis, verify it, and only then consider AI assistance for the fix itself. Debugging skill is the most valuable skill an engineer has. It's also one of the first to atrophy when AI is available for every error message. **Post-mortem AI audit.** When a bug reaches production that was introduced by AI-generated code, include in the post-mortem: what should the review process have caught? What could the author have caught? What will change? This creates learning from failures rather than quietly fixing and moving on. ### Ownership Culture The accountability gap in AI code review isn't just a process problem — it's a culture problem. If the team's mental model is "the AI wrote it, so it's the AI's fault when it breaks," the culture is wrong and the engineering quality will reflect that. Building ownership culture requires consistent messaging at every level: **In PR descriptions.** Engineers should write as though they wrote the code, because they own it. "I implemented X using AI assistance and verified Y" is the right framing, not "AI wrote this." **In post-mortems.** When AI-generated code fails, the post-mortem should analyze the engineering decision to ship it, not just the AI's output. Who reviewed it? What was verified? What was missed? The answers should inform process, not assign blame. **In management language.** Engineering managers set the tone. If a manager says "well, the AI generated that bug" in a post-mortem, the message is that AI-generated code is a different category of responsibility. That's the wrong message. The engineer owns the code. Full stop. > **Warning** > Teams that develop a two-tier ownership model — "code we wrote" and "code AI wrote" — will have inconsistent quality and unclear accountability. Every line in the codebase is owned by the engineering team. The tool that produced it is irrelevant to responsibility. ### Preserving Code Comprehension Across the Team In a codebase where AI writes significant code, the risk of diffuse incomprehension is real. Individual engineers understand the code they were personally involved with and vaguely understand everything else. If the code was AI-generated and reviewed quickly, even the authors may have shallow understanding. Practices that maintain collective comprehension: **Architecture decision records (ADRs).** For significant design decisions — even ones that emerge from AI-assisted development — write brief ADRs. Not because the AI made the decision, but because the decision was made in the context of AI output and may not be obvious from the code. **Module ownership with accountability.** Assign module ownership clearly. The owner of a module should be able to explain what it does, why it's structured the way it is, and what its key invariants are. If the module is primarily AI-generated and the owner can't explain it, that's a gap to address. **Reading group for complex AI-generated systems.** When a significant system component is built with heavy AI assistance, schedule a team reading session where the code is walked through collaboratively. Not to critique — to build shared understanding. Teams that skip this step end up with systems that only one person understands, and that one person may not have written a line of it. ### The Productivity Trap AI tools can create a productivity incentive structure that works against code quality. If engineers are measured on features shipped and PRs merged, and AI tools increase both, engineers who use AI tools most aggressively are rewarded by the metrics. Engineers who use them carefully — verifying output, writing human tests, reviewing thoroughly — may produce fewer metrics. This is a management problem, not a technology problem. Incentive structures determine behavior. If the incentive is velocity, engineers optimize for velocity. If the incentive is quality, they optimize for quality. AI tools make this tension more acute because the gap between "ship fast and let it accumulate" and "verify and own it" is larger. Addressing this requires explicitly rewarding quality-related behaviors: - Catching significant issues in review (not just leaving nit comments) - Adding tests that catch real bugs - Improving documentation and comprehensibility of AI-generated code - Raising flags on AI-generated code that doesn't fit the system These behaviors should appear in performance conversations if they're going to compete with raw velocity metrics. ### Integrating New Engineers Onboarding new engineers onto a team with heavy AI usage requires explicit calibration. New engineers who see senior engineers using AI tools extensively may assume this is the default workflow for everything. They may not see the foundation of understanding that makes senior-engineer AI usage safe and junior-engineer AI usage risky. Onboarding for AI-code teams should explicitly cover: - Where AI tools are used and where they aren't (and why) - The review standards and checklists - How to verify AI output before submitting - The team's ownership culture - War stories: times AI-generated code failed and what that looked like That last item matters. New engineers have no mental model for how AI code fails. Hearing concrete stories of failures — and what they cost — is more effective than abstract warnings. ### The Long-Term Craft Question There is a legitimate and unresolved question about what engineering craft looks like in five years when AI writes the majority of code. The answer isn't obvious. What is clear is that the teams that will do best are the ones that deliberately maintain the human skills that complement AI: system thinking, requirements clarification, architecture, diagnosis, security intuition, business context. These are the skills AI can't replicate because they require understanding of the specific system, the specific business, and the specific constraints. They're also the skills that atrophy fastest if teams stop exercising them in favor of prompt-first engineering. The practical implication: treat AI tools as automating the mechanical parts of coding, and invest in the judgment parts. Pair programming focused on architectural decisions. Post-mortems focused on diagnosis and reasoning. Code review focused on system fit. These are the high-value activities that AI doesn't automate and that should get more attention, not less, as AI handles more mechanical code. > **Try This** > Conduct a team retrospective specifically about AI tool usage. Ask three questions: Where is AI saving real time and producing good output? Where is AI producing work we're not confident in? Where are we using AI when we'd be better served by thinking through the problem ourselves? The answers to those three questions are a more honest picture of your current situation than any metric dashboard. ### Key Takeaways 1. Deskilling risk is real and lags — it appears slowly and compounds before it becomes visible. 2. Intentional learning structures (manual implementation, code walkthroughs, debugging practice) counteract deskilling. 3. Ownership culture must be explicit and consistent — AI tool usage doesn't change responsibility for the code. 4. Incentive structures need to reward quality behaviors alongside velocity, or AI tools will be used in ways that maximize throughput at the expense of quality. 5. The skills that complement AI — system thinking, diagnosis, architecture, security intuition — are the ones to invest in as AI handles more mechanical coding. --- ## Conclusion The central challenge with AI-generated code is not technical. The technology works. AI tools write functional, syntactically correct code at high velocity. The challenge is that they do so without context, without stakes, and without the accumulated judgment that makes code fit for the specific system it's going to run in. Teams that treat AI tools as fast typists — producing code that needs to be understood and owned like any other code — will capture the productivity gains while managing the risks. Teams that treat AI tools as a wholesale substitution for engineering judgment will accumulate a debt that surfaces as production incidents, security vulnerabilities, and a codebase nobody fully understands. The processes described in this book are not burdensome if built correctly. A review checklist takes thirty seconds to use. Automated SAST gates run without human involvement. Attribution labels on PRs are three clicks. The investment is small. The value is in the discipline: every engineer on the team understanding what AI code is, what it requires, and what they're responsible for. A few things this book has not resolved, because they're genuinely unresolved: How AI tools will change what "senior engineer" means in five years is uncertain. The mechanical parts of coding will be largely automated. The judgment parts — understanding systems, evaluating requirements, making architectural decisions under constraint — seem durable. But the path from junior to senior has traditionally run through the mechanical parts. The learning path for the next generation of engineers is going to need reinvention. Whether current AI coding tools are at a quality ceiling or will substantially improve is also uncertain. Better context handling, better system understanding, and better uncertainty calibration would address most of the failure modes described here. Some of that improvement is coming. Planning for it is reasonable. Betting production quality on it before it arrives is not. What remains constant: code in production must be correct, secure, and maintainable. It must be owned by engineers who understand it. The review process must be designed to verify those properties, not to rubber-stamp output that looks correct. Those requirements don't change with the technology. Use the tools. Understand their failure modes. Build processes that catch what they can't catch. Own the output. --- ## Appendix A: Glossary **Aligned failure** — When AI-generated code and AI-generated tests are both wrong in the same direction, causing all tests to pass for incorrect behavior. The tests confirm the code's interpretation rather than an independent specification. **Attribution** — The practice of tagging code changes with their origin (AI-generated, AI-assisted, human-written) to enable quality metric segmentation by source. **Context blindness** — The failure mode where AI-generated code is correct in isolation but wrong in the context of a specific codebase, because the model lacks knowledge of the codebase's conventions, existing abstractions, and accumulated decisions. **Dependency confusion attack** — A supply chain attack where a malicious package is published under a name that conflicts with an internal package name, causing package managers to download the malicious version. **Deskilling** — The gradual loss of competency in a skill due to reduced practice. In engineering, AI tools that handle mechanical coding tasks can reduce the practice of those tasks for early-career engineers. **Gate failure rate** — The percentage of code changes that fail automated quality gates (SAST, dependency scanning, secret scanning, type checking) in the CI/CD pipeline. **Mutation testing** — A testing technique that deliberately introduces small bugs (mutations) into code and measures whether the existing tests catch them. Produces a mutation score indicating test quality. **Overly permissive defaults** — A pattern in AI-generated code where security configurations are set to maximally permissive values to ensure functionality, rather than to least-privilege values appropriate for production. **Post-merge defect density** — The number of bugs found after code is merged to the main branch, per unit of code changed. A lagging indicator of code quality. **Prompt injection** — An attack where malicious content in data processed by an AI model contains instructions that alter the model's behavior. In agentic coding contexts, can influence code generated by AI tools that read external data. **Property-based testing** — A testing approach where tests specify properties that should always be true (rather than specific input/output pairs), and a framework generates test cases to verify those properties. Useful for AI-generated code because properties can be human-specified while test cases are generated automatically. **Review discovery rate** — The fraction of bugs in a cohort of code that were caught during code review, as opposed to QA or production. A measure of review effectiveness. **SAST (Static Application Security Testing)** — Automated code analysis tools that scan source code for security vulnerabilities without executing the code. Examples: Semgrep, Bandit, SonarQube. **Supply chain attack** — An attack that targets the software supply chain — dependencies, build tools, registries — rather than application code directly. AI-recommended dependencies that are malicious or compromised are a supply chain risk vector. **Training data artifacts** — Patterns in AI-generated code that reflect historical coding practices from training data, including deprecated APIs, outdated security patterns, or conventions from older versions of frameworks. **Transitive dependency** — A package that is included not because the project directly depends on it, but because a package the project depends on requires it. Transitive dependencies multiply the security and license risk of any new direct dependency. --- ## Appendix B: Tools & Resources ### Static Analysis and Security Scanning **Semgrep** — Open-source, multi-language SAST tool with an extensive rule library. Supports custom rules. Free for open-source use. Integrates with CI/CD pipelines. Strong for catching AI-generated security antipatterns. `semgrep.dev` **Bandit** — Python-specific security linter. Fast, opinionated, good default rule set for common Python security issues. `github.com/PyCQA/bandit` **ESLint with security plugins** — For JavaScript/TypeScript. The `eslint-plugin-security` plugin adds security-specific rules to standard ESLint. Integrates naturally into existing JS workflows. **SonarQube** — Comprehensive code quality and security platform. Commercial with free community edition. Broad language support. More overhead to set up but broader coverage than lighter tools. ### Dependency Scanning **Snyk** — Commercial vulnerability and license scanning for dependencies. Strong ecosystem support (Node, Python, Java, Go). CI/CD integration. Free tier available for open-source. **pip-audit** — Python dependency vulnerability scanner using PyPI advisory database. Simple CLI, fast, good for CI integration. **npm audit** — Built into npm. Scans `package-lock.json` against the npm security advisory database. Run as a CI gate. **OSV-Scanner** — Google's open-source vulnerability scanner using the Open Source Vulnerabilities (OSV) database. Multi-ecosystem support. **FOSSA** — License compliance scanning with commercial support. Produces actionable license reports for commercial software. **license-checker** — Node.js license scanner. Simple, fast, good for auditing npm dependency licenses. ### Secret Detection **gitleaks** — Open-source secret scanner for git repositories. Scans both current code and git history. Fast and comprehensive rule set. CI/CD integration available. **truffleHog** — Secret scanner with entropy analysis and regex rules. Strong at finding high-entropy strings that might be credentials even without matching known patterns. **detect-secrets** — Yelp's secret detection tool. Has a baseline file concept — good for projects that already have known false positives to exclude. ### Testing Tools **Hypothesis** — Python property-based testing library. Excellent for testing AI-generated code with human-specified properties and generated test cases. **mutmut** — Python mutation testing tool. Produces clear reports on which mutations weren't caught by the test suite. **Stryker** — Mutation testing framework for JavaScript, TypeScript, and more. Active development, good reporting. **PIT (PITest)** — Java mutation testing. Fast and widely used in Java ecosystems. ### Code Review and Process **GitHub CODEOWNERS** — File-based code ownership specification. Routes PRs to appropriate reviewers automatically. Key tool for ensuring AI-generated code in specific areas goes to engineers with context in those areas. **Danger** — Automated code review feedback tool. Can enforce PR conventions, check for required labels, flag missing checklist items. Useful for lightweight enforcement of AI-code review process. **Reviewpad** — GitHub workflow automation for code review. Can enforce reviewer routing rules, required checklist completion, and label-based review gates. ### Dependency Management **Renovate** — Automated dependency update PRs. Essential for staying current on security patches across all dependencies, including those added by AI-generated code. **Dependabot** — GitHub's built-in dependency update tool. Simpler than Renovate, good default choice for GitHub-hosted projects. --- ## Appendix C: Further Reading ### On Code Review **"Code Review Best Practices"** — Karl Wiegers. Original thinking on what code review is for and how to make it valuable. Predates AI tools but the fundamental principles apply and the contrast with AI-code review is clarifying. **"How Google Does Code Review"** — Various Google Engineering practices documents, publicly available. The culture of code review as education and shared ownership is particularly relevant. **"What We Know About Formal Inspections"** — Software Engineering Institute literature on inspection-based code review. The empirical data on review effectiveness versus automated testing is useful for calibrating where to invest review effort. ### On AI Coding Tools **GitHub Copilot Research** — GitHub has published several research papers on Copilot's effects on developer productivity and code quality. Methodologically imperfect but the only large-scale longitudinal data available. Worth reading with a critical eye. **"Evaluating Large Language Models Trained on Code"** — Chen et al. (2021). The Codex paper that underpins much of current AI coding tool development. Describes both capabilities and failure modes that remain relevant. **"Security Weaknesses of Copilot Generated Code in GitHub"** — Pearce et al. (2022). Systematic analysis of security issues in Copilot-generated code across multiple vulnerability categories. Concrete and well-documented. ### On Software Quality and Testing **"A Philosophy of Software Design"** — John Ousterhout. The best current book on managing complexity in software. Its framework for deep vs. shallow modules, and for complexity accumulation, is directly applicable to managing AI-generated code. **"Growing Object-Oriented Software, Guided by Tests"** — Freeman and Pryce. The TDD philosophy here — tests as specifications, not just verification — directly addresses the aligned failure problem. The "what should this do?" framing for tests is the right mental model for testing AI-generated code. **"An Introduction to Property-Based Testing"** — Various. The Hypothesis documentation and accompanying blog posts by its author (David MacIver) are the best introduction to property-based testing philosophy. Start with "In Praise of Property-Based Testing" on the Increment blog. ### On Security **OWASP Top 10** — The standard catalog of application security risks. The categories (injection, broken access control, cryptographic failures) map directly to AI-generated code failure modes. **"The Web Application Hacker's Handbook"** — Stuttard and Pinto. Detailed treatment of web application vulnerabilities. Understanding how these vulnerabilities work is necessary for recognizing AI-generated code that introduces them. **"Threat Modeling: Designing for Security"** — Adam Shostack. Systematic approach to threat modeling. The process of asking "what could go wrong?" before writing code is particularly valuable as a complement to AI code generation, where the model doesn't ask that question. ### On Engineering Culture **"An Elegant Puzzle: Systems of Engineering Management"** — Will Larson. The chapters on engineering process and technical quality are directly relevant. The framework for managing technical debt applies to the accumulated quality debt from unreviewed AI-generated code. **"The Pragmatic Programmer"** — Hunt and Thomas. The foundational thinking on code ownership, understanding your tools, and taking responsibility for output hasn't been superseded by AI tools. Read it with the question: what does it mean to "know your tools" when one of your tools is an AI model? --- *AI-Generated Code: Quality, Risk, and Review* *Version 1.0 — April 2026* *David Kelly Price* *Pyckle — AI/ML Tooling, Retrieval Systems, Context Optimization* ``` The agent produced the full missing content. Here is the complete continuation, ready to append directly to the existing ebook: --- --- *© 2026 Pyckle. All rights reserved. This guide may be shared freely for personal and educational use. Commercial reproduction or redistribution requires written permission. Contact kellyprice@pyckle.co.* --- ## Related Blog Posts - [When AI Writes Itself: What 100% AI-Generated Code Actually Means](https://pyckle.co/blog/when-ai-writes-itself-what-100-percent-ai-generated-code-actually-means.html) - [Apple Brings Agentic Coding to Xcode](https://pyckle.co/blog/apple-brings-agentic-coding-to-xcode-the-real-question-is-what-happens-next.html) --- *[Browse all free guides →](https://pyckle.co/books.html)*