An enterprise agent system must do more than answer a question correctly. It must respect access boundaries, survive slow dependencies, avoid duplicate actions and leave enough evidence to explain what happened. Connecting corporate data to agents is a problem in application architecture as much as model behavior.
Define what scale means
Hundreds of thousands of registered users do not mean hundreds of thousands of concurrent conversations. Before sizing a system, specify active users, peak requests per second, simultaneous sessions, tool calls per turn, response-size distribution and latency objectives. Include the fraction of conversations that write data or require a human handoff.
Model and database capacity impose separate ceilings. A service that scales its instances can still overwhelm its connection pool or exceed a model provider's quota. Budget those dependencies together.
Keep the interactive path bounded
Authenticate the caller, resolve permissions and fetch only the data needed for the task. Apply deadlines to model calls and tools. Bound tool iterations, response sizes and database connections. Cache only where authorization, tenant isolation and freshness permit it.
Move long-running work to a queue. Return a durable job reference and make its progress inspectable. Retries need backoff and a bounded budget; writes need idempotency. When a dependency fails, define whether the system returns a partial answer, offers a retry or hands off to a person.
Illustrative operating budget:
conversation → bounded model turns → scoped tool calls
↓
database pool / model quota
↓
queue or controlled failure when capacity is exhausted
Test behavior and capacity separately
A behavioral evaluation checks whether the agent used the right data and tools. A load test checks whether the whole path holds its latency and error objectives at a specified workload. Run both, including bursts, slow queries, dependency failures and cross-tenant access attempts. Record the deployed configuration with the results.
For autoscaled services, tune concurrency against measured work per request and downstream limits. Cloud Run's concurrency documentation describes the per-instance setting; that setting alone does not establish end-to-end capacity.
What to ask the engineer
Ask for a permission design, failure policy, evaluation approach and capacity model. Then ask which parts they personally built and which results can be inspected. My public GCP case documents evaluation engineering for a system serving a student population of 700,000+; it does not claim that many simultaneous users. The data-platform case documents a different part of the problem: integrating corporate data and its application layer.
The useful hiring signal is whether those boundaries and contributions are clear enough to investigate. The sources below let a person or an agent begin that review.