Is my vibe coded app secure? A 30-minute test plan you can run today

Updated Oct 22, 20269 min read

Is my vibe coded app secure? Not reliably, and in 30 minutes you can find the most common holes. Run six tests on your own app: logged out, second account, secrets search, view source, direct database access and abuse. Each test has a red result (you have a problem) and a green result (it passed). Fix the reds, then decide whether you need a human audit.

Is my vibe coded app secure? What the research says

Veracode tested more than 100 large language models on coding tasks in Java, Python, C# and JavaScript. In the Veracode 2025 GenAI Code Security Report, 45% of code samples failed security tests and introduced OWASP Top 10 vulnerabilities. JavaScript samples failed 43% of the time, and defenses against cross-site scripting (XSS, where an attacker's code runs in another user's browser) were missing in 86% of the relevant samples.

Newer models didn't fix it. Veracode reports that models got better at writing code that works and stayed flat on writing code that's secure, whatever their size or release date.

Vibe coding security risks follow the standard web list. The OWASP Top 10:2025 puts Broken Access Control (users reaching data or actions they shouldn't) first and Security Misconfiguration (unsafe settings) second. Tests 1, 2 and 5 below check access control. Tests 3 and 4 look for secrets sitting where they shouldn't.

For a longer review list, use the vibe coding security checklist. For the wider question behind it, read is AI-generated code secure.

How attackers check an AI-built app

Most attackers don't start with your source code. They open your site, watch what the browser downloads, and try the obvious moves:

• Load private pages and API URLs while logged out.
• Change an ID in a request to see someone else's record.
• Search the downloaded JavaScript for keys.
• Use the public database key to talk to the database directly.
• Send the same request hundreds of times.

If your repo is public, they read that too. Your six tests copy these moves. A demo shows one user clicking the buttons you designed, so an app can work perfectly and still fail every test below.

Test it yourself: six tests in 30 minutes

Run these on your own app, ideally a test copy with throwaway accounts. Replace the placeholders with your own values. The times add up to 30 minutes.

Test 1, logged out (5 minutes). Open a private window so you have no session and visit every private page, such as /dashboard and /admin. Then replay the API calls those pages make. Find them in DevTools (F12) under the Network tab.

curl -i https://YOUR-APP.com/api/invoices/123

Red: a 200 response with real data, or a page that shows content before it redirects. Green: a 401, a 403, or a redirect to login with no data in the response.

Test 2, second account (5 minutes). Sign up two users, A and B. As A, create something private and copy its ID from the URL or the Network tab. Log in as B in another browser profile and request that ID, then try to edit and delete it.

Red: B can read, change or delete A's record. Green: B gets a 403, a 404 or an empty result. This bug is called IDOR, and the IDOR guide shows the fix.

Test 3, secrets in your code (5 minutes). Run these in your repo. A service_role hit that is only a variable name is fine. A long string starting with eyJ is a JWT, and the Supabase service role key is one. The last command searches git history, where deleted keys survive.

git grep -nIE 'sk_live_|sk_test_|sb_secret_|service_role|AKIA|BEGIN [A-Z ]*PRIVATE KEY|eyJ[A-Za-z0-9_-]{20,}\.'
git grep -nIE 'NEXT_PUBLIC_[A-Z_]*(SECRET|SERVICE|PRIVATE)'
git ls-files | grep -E '(^|/)\.env'
git log --all --oneline -G'sk_live_|sb_secret_|service_role|AKIA|PRIVATE KEY'

Red: a real key in source, a tracked .env file, or a hit in history. A key that ever reached history is compromised, so rotate it as described in the leaked API key guide. Green: only placeholders, such as a committed .env.example.

Test 4, view source (5 minutes). Open your live site, click through a few pages, then press Ctrl+Shift+F (Cmd+Option+F on Mac) in DevTools to search every loaded file. Search for sk_, sb_secret_, service_role, secret, Bearer and eyJ. Paste any eyJ string into a JWT decoder. If the payload says "role": "service_role", it's your secret key.

Red: any private key, service role key or third-party secret. Green: only public values, such as your Supabase URL, the anon key and a Stripe publishable key (pk_).

Test 5, database with the anon key (7 minutes). Take your project URL and the anon key (the public key your frontend uses to reach Supabase) from your frontend code and ask the database for rows directly. Newer Supabase projects may call it a publishable key, so check the current Supabase docs for the exact headers.

curl "https://YOUR-PROJECT.supabase.co/rest/v1/profiles?select=*&limit=5" \
  -H "apikey: YOUR_ANON_KEY" \
  -H "Authorization: Bearer YOUR_ANON_KEY"

Run it against every table that holds private data. Red: rows come back. Green: a permission error, or an empty array for a table you know has rows. An empty array from an empty table proves nothing, so check in the dashboard.

That only tests the logged-out role. A bad policy can let any signed-in user read every row, so repeat the request with user B's access token in the Authorization header. Red: B sees A's rows.

Then try POST, PATCH and DELETE against a throwaway row, adding -H "Prefer: return=representation". An empty [] on PATCH or DELETE means RLS blocked it. Confirm in the dashboard whether the row changed. If anything succeeds, read Supabase RLS mistakes.

Test 6, abuse (3 minutes). Send 30 wrong logins to your own test account and count the status codes. Point the loop at the endpoint your login form really calls (copy it from the Network tab). The URL below is a placeholder.

for i in $(seq 1 30); do
  curl -s -o /dev/null -w "%{http_code}\n" -X POST https://YOUR-APP.com/api/login \
    -H "Content-Type: application/json" \
    -d '{"email":"you@example.com","password":"wrong-password"}'
done | sort | uniq -c

• Red: 30 identical 401 or 400 responses and no 429 ("too many requests"). Thirty 404s mean you hit the wrong URL.
• Green: a 429, a lockout or a captcha before the 30th try.

Hosted auth providers often add their own limits, so check their docs first. Then call your costliest route, such as an AI or email endpoint, 30 times and look at what your provider billed. The login rate limiting guide covers the fix.

What a failing test looks like in code

Tests 1 and 2 usually fail for the same reason: the route trusts the request. The first handler below uses the service role key, an admin key that skips row level security (RLS, the database rules that decide which rows each user can read or change), and never asks who is calling.

// app/api/invoices/[id]/route.ts
import { supabaseAdmin } from "@/lib/supabase-admin" // service role key

export async function GET(
  _req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const { id } = await params
  const { data } = await supabaseAdmin
    .from("invoices")
    .select("*")
    .eq("id", id)
    .single()
  return Response.json(data)
}

// app/api/invoices/[id]/route.ts
import { createClient } from "@/lib/supabase/server" // anon key + user cookies

export async function GET(
  _req: Request,
  { params }: { params: Promise<{ id: string }> }
) {
  const supabase = await createClient()
  const { data: { user } } = await supabase.auth.getUser()
  if (!user) return new Response("Unauthorized", { status: 401 })

  const { id } = await params
  const { data } = await supabase
    .from("invoices")
    .select("*")
    .eq("id", id)
    .eq("user_id", user.id)
    .maybeSingle()

  if (!data) return new Response("Not found", { status: 404 })
  return Response.json(data)
}

The fixed version asks Supabase who the caller is with getUser(), which checks the token with the Auth server. It then returns only rows owned by that user.

How to fix what the tests find, step by step

Fix in this order. Leaked secrets come first because they're usable the moment someone sees them.

1. Rotate leaked keys first

   If test 3 or 4 found a real secret, revoke it at the provider and issue a new one. Do this before any code cleanup, because anyone who saw the key can still use it.

2. Check the caller on the server

   For every route and server action that failed test 1 or 2, read the session with getUser() and filter queries by that user's ID. Never trust a user ID sent from the browser.

3. Turn on row level security

   For every table that failed test 5, enable RLS and add a policy for each action you need: select, insert, update and delete. Then repeat the curl test.

4. Keep private keys on the server

   Put secrets in server-only environment variables. Anything with a NEXT_PUBLIC_ prefix goes to every visitor.

5. Add rate limits

   Limit login, signup, password reset and any route that costs money or sends email. Return a 429 when the limit is hit.

6. Run all six tests again

   Save the date and the results. Re-run after any change to auth, database policies or routes your AI tool adds.

Fix prompt you can paste

This prompt makes your AI tool find the code behind each test. It audits first and edits only after you approve.

Audit this repo for the code behind six security tests. Do not change anything yet.

1. List every API route (app/api/**/route.ts) and server action ("use server") that reads or writes user data. Say whether each checks the session with supabase.auth.getUser() (or the Clerk/NextAuth equivalent) and filters by the current user's ID. Flag any route that trusts an ID sent from the client.
2. Find every use of the Supabase service role key. Flag any that a request can reach without an auth check.
3. Search for hardcoded secrets (sk_live_, sk_test_, service_role, private keys, AKIA), NEXT_PUBLIC_ variables that hold secrets, .env files tracked by git, and secrets in git history. Flag private pages protected only by a client-side redirect. Never print secret values. Show only file:line and the first 4 characters.
4. List every table in migrations or SQL files and say whether RLS is enabled and which policies exist for select, insert, update and delete.
5. List login, signup, password reset and every route that calls a paid API or sends email, and say whether each has rate limiting.

Output a table with file:line, problem, severity and suggested fix. Explain in plain words what each fix would change. List every key that must be rotated. Wait for my approval before editing any file.

What secure enough means, and when to hire a human

Secure enough depends on what an attacker gets if they win. Here's our rule of thumb. It's our opinion, and no standard backs it.

• A hobby app with no accounts needs tests 3 and 4, plus test 5 if it uses a database and test 6 on any route that calls a paid API.
• Once users can log in, you need all six.
• Once the app takes payments or stores other people's personal data, you also owe those users a second pair of eyes.

Passing six tests means you've closed the doors that curious strangers and automated scans try first. It doesn't prove the app is secure. The tests skip three areas:

• Dependencies, the packages your app installs. Run npm audit and check that every package your AI added is real, because AI tools sometimes make up package names.
• Prompt injection, where text a user supplies tricks an AI feature into ignoring its instructions.
• The permissions you give your coding agent.

OWASP's Secure Coding with AI cheat sheet covers all three.

Get a human audit when:

• You take payments or store sensitive personal data.
• A customer, investor or partner asks for a pen test (a hired expert's simulated attack on your app) or a security questionnaire.
• You have teams, organizations or roles, where one broken permission check exposes one customer's data to another.
• You have an admin panel, uploads, or features that fetch URLs users supply.
• A test failed and you can't explain why the fix works.

A human also checks business logic, such as whether a coupon works twice or a refund can be triggered by the wrong user. Automated tools rarely catch those. You can run a Leakcade scan first so a paid auditor's time goes to logic.

Further reading

1. Veracode: 2025 GenAI Code Security Report
2. OWASP Top 10:2025
3. OWASP Cheat Sheet: Secure Coding with AI

FAQ