GuidesOneMux

Secure OpenAI API Keys in a Document-Q&A Workflow

A practical security pattern for OpenAI document question answering: separate credentials, restrict permissions, redact logs, and rotate keys safely while preserving source attribution.

AI

Summary · TL;DR

In a document-Q&A application, keep the OpenAI application credential in a trusted server-side secret store, not in a browser, sandbox, source file, or log. Give each workload only the access it needs, redact sensitive data from logs, and rotate credentials using a documented overlap-and-revoke procedure. Return source references with each answer so users can verify claims against retrieved documents.

Key Takeaways

  • Keep the OpenAI application key outside agent-generated code and other untrusted execution environments.
  • Use a dedicated project or workload boundary, and grant only the API permissions the application requires.
  • Log request IDs, timings, and source identifiers rather than credentials, document passages, or full prompts.
  • Return document identifiers and page or section references alongside answers; do not treat a fluent answer as proof.
  • Rotate by deploying a replacement credential, verifying traffic, then revoking the old one; revoke immediately if exposure is suspected.
On this page (6)

A document-Q&A system needs to do two things well: answer from the right material and keep its credentials out of the material-processing path. The safest default is to have a trusted application server call OpenAI, while retrieval and citation handling remain explicit parts of your own workflow. Do not place the application key in a browser, document, prompt, source repository, or agent-accessible sandbox.

Start with a trust boundary

A typical flow is: a user asks a question; your application retrieves relevant passages from an authorized document store; the server sends the question and selected passages to OpenAI; the application returns the answer with references to those passages. The key belongs on the server side, in a managed secret store or equivalent protected configuration—not in code sent to a user or a workload that can inspect its environment.

OpenAI’s sandbox security guidance warns that agent-generated code can access files, credentials, and network resources available to its environment. It recommends isolated workloads, separate environments where data must not be shared, dedicated OpenAI projects for applications or workloads, and keeping the application API key outside the environment. It also says not to embed keys in images, source code, or logs. See OpenAI sandbox security guidance.

For document Q&A, that means treating uploaded documents and model-generated code as untrusted inputs. A retrieval worker should receive only the documents and tools needed for its task. It should not inherit an unrestricted application credential simply because it needs to send a question for inference.

Limit permissions and separate credentials

Create a separate project or credential boundary for this application rather than sharing a broad-purpose key across unrelated workloads. Apply the narrowest permissions available for the API operations your implementation actually uses. OpenAI’s cited guidance specifies api.agents.read and api.agents.write for sessions, api.responses.write for inference, and api.vaults.read and api.vaults.write for vault management. Those are examples for the documented Agents API use cases; do not grant every permission by default. Confirm the permissions required by the APIs you actually call.

If an executor environment needs to connect to OpenAI, the guide describes using a separate environment key for that connection. It says this key permits connecting environments and cannot authorize other API actions. Keep that distinct from the application key. For third-party services, keep credentials in the application or a trusted proxy; do not inject long-lived secrets into an environment where generated code can read them.

Redact logs without losing operational value

Logs help diagnose empty retrievals, failed requests, and missing citations, but full prompts and documents can contain personal, confidential, or commercially sensitive information. Log only what your team needs to operate the service, for example:

  • A request ID, timestamp, status, and latency.
  • The model and application version, if useful for debugging.
  • Internal document IDs and page or section references, rather than full retrieved text.
  • A redacted error category, not request headers or secret values.

Never log authorization headers, API keys, full user prompts, or complete document passages by default. If you need temporary content-level debugging, use a controlled, time-limited process with access restrictions and a clear deletion date. Check application, proxy, tracing, and error-reporting systems too: redaction at the main application logger does not automatically sanitize other capture points.

An illustrative logging pattern is:

log_event(
    request_id="generated-request-id",
    outcome="success",
    retrieved_sources=["policy-17:p4", "handbook-03:section-2"],
    credential="[never include]",
    prompt="[omit or redact]",
    passage_text="[omit or redact]"
)

This is pseudocode, not a tested OpenAI SDK example. Adapt it to your logging system and verify that secrets are removed before events leave the application process.

Make source attribution part of the answer contract

Do not ask the model to invent citations after the fact. Preserve the source metadata during retrieval, associate each passage with its document ID and page, section, or URL, and provide that metadata to the answer-generation step. Require the response format your application expects: an answer plus references drawn only from the retrieved set. Then validate references in application code before displaying them.

If retrieval returns no relevant passages, say that the available documents do not support an answer or ask the user to narrow the question. If the answer contains a claim that cannot be connected to a returned source, mark it as unsupported or omit it. A citation is useful only if the referenced location exists and contains evidence for the claim. The exact retrieval and response-format implementation depends on your chosen APIs; the security guidance cited here does not prescribe a document-Q&A or citation implementation.

Define a credential rotation procedure

Write the steps down before an incident. A practical routine is:

  1. Identify the application, project, secret-store entry, and services using the current credential.
  2. Create a replacement credential with the same minimal required access, and store it in the approved secret store.
  3. Deploy the replacement through a controlled configuration change. Avoid printing it during deployment or placing it in build artifacts.
  4. Send a test question through the normal workflow. Confirm inference succeeds, citations still resolve, and logs contain no secret or document text.
  5. Monitor errors, then revoke the old credential once the replacement is confirmed in use. Remove stale copies from deployment systems and local development environments.
  6. If exposure is suspected, revoke the affected credential immediately, investigate access and logs, and issue a replacement. Do not wait for the regular rotation window.

OpenAI’s guidance says to rotate or revoke keys when needed and to revoke credentials immediately if exposure is suspected. Your organization should set its own rotation interval based on risk, policy, and operational constraints; the cited source does not prescribe a universal schedule.

Checklist before launch

  • The application key is server-side and inaccessible to browsers and agent-generated code.
  • The workload is isolated appropriately, with a dedicated project or environment where warranted.
  • Permissions are limited to the operations the application needs.
  • Logs exclude keys, authorization headers, raw prompts, and full retrieved passages.
  • Answers include validated references to retrieved documents.
  • A named owner can deploy a replacement credential and revoke the old one.

The core design choice is simple: keep credentials outside the document-processing environment, and make citations traceable to retrieved sources. That reduces the impact of a compromised workload while giving users a way to check what the answer relies on.

FAQ

Can I put an OpenAI API key in a browser-based document-Q&A app?

Do not expose the application key to browser code. Make OpenAI requests from a trusted server-side component that retrieves the key from protected configuration or a secrets manager.

Should I log prompts and retrieved document passages to debug answers?

Avoid doing so by default. Prefer request IDs, status, timing, and source identifiers. If temporary content-level debugging is necessary, restrict access, set a deletion deadline, and ensure credentials are never captured.

How often should OpenAI API keys be rotated?

The cited OpenAI security guidance says to rotate or revoke keys when needed but does not specify a universal interval. Set a schedule based on your organization's risk and policy, and revoke immediately if exposure is suspected.

How do I make document-Q&A citations trustworthy?

Carry document IDs and page, section, or URL metadata through retrieval, restrict references to the retrieved source set, and validate each reference before displaying it. Do not rely on the model to invent citations.

Related articles