Introduction
Amazon Bedrock is a managed service that provides access to foundation models through a single programming interface. A foundation model is a large model that has been trained on a broad range of data and can be adapted to many tasks. The generation of new text or other content by such a model is referred to as generative AI. This guide explains what Bedrock is, why it exists, how it operates, and the situations in which it is appropriate.
What is it?
Bedrock offers a selection of foundation models from Amazon and from third-party providers, all reached through one interface. Because the models are provided as a managed service, no servers are operated by the customer, and a different model can be selected without the application being rewritten. In addition to the models themselves, Bedrock provides managed features for grounding answers in private data, for allowing a model to take actions, and for applying safety controls.
Why does it exist?
Running a large foundation model requires specialized hardware, considerable operational effort, and careful attention to security. Few organizations are able to host such models efficiently. Bedrock was created so that these models can be used through a simple interface, without infrastructure being managed, and with the data kept within the customer's control. This lowers the barrier to building applications that use generative AI.
How it works
An application sends a prompt to a model through the Bedrock interface. A prompt is the input text that instructs the model. The model performs inference, which is the process of producing an output from an input, and returns a completion, which is the generated response. Both the prompt and the completion are measured in tokens, where a token is a small unit of text of roughly a few characters. A Knowledge Base can be attached so that answers are grounded in a set of documents, and an Agent can be configured so that the model calls tools and takes actions. Guardrails can be applied to filter unwanted content.
Architecture diagram
Advantages
- Managed access. Foundation models are used without infrastructure being operated.
- Choice of models. Several models are available through one interface.
- Data control. The data that is sent is not used to train the underlying models.
- Integrated features. Grounding, actions, and safety controls are provided.
- Serverless. Capacity scales with demand and idle capacity is not paid for.
Disadvantages
- Cost per token. Large volumes of text can become expensive.
- Model limits. Each model has a maximum amount of text it can consider at once.
- Regional availability. Not every model is available in every region.
- Less control. The internal behavior of a model cannot be changed as fully as a self-hosted model.
Common use cases
- Conversational assistants that answer questions in natural language.
- Summarization of long documents.
- Question answering grounded in private documents through a Knowledge Base.
- Agents that perform tasks by calling tools.
Best practices
- The model should be chosen to suit the task, balancing quality against cost.
- Prompts should be written carefully, a practice known as prompt engineering, so that the output is reliable.
- Guardrails should be applied to filter unwanted or unsafe content.
- Token usage and cost should be monitored, because charges depend on the amount of text.
Common mistakes
- A model that is more powerful and expensive than the task requires is chosen.
- Cost controls are not applied, so token usage grows unexpectedly.
- Guardrails are omitted, which allows unwanted content to be produced.
- Answers are expected to be current when the model was trained on older data, without a Knowledge Base being used.
Related AWS services
- AWS Lambda calls Bedrock and provides the tools that an Agent uses.
- Amazon S3 stores the documents that a Knowledge Base indexes.
- AWS Step Functions can coordinate the steps of a larger AI workflow.
Frequently Asked Questions
- What is a foundation model?
- A foundation model is a large model trained on a broad range of data that can be adapted to many tasks, such as generating text, answering questions, or summarizing documents.
- Which models are available in Amazon Bedrock?
- Bedrock offers models from Amazon and from several third-party providers. The list changes over time and should be checked in the service. The single interface allows a model to be changed without the application being rewritten.
- Is customer data used to train the models?
- The data sent to a model in Bedrock is not used to train the underlying foundation models, and it remains within the customer's control, subject to the current terms.
- How is Amazon Bedrock priced?
- Charges are based on the number of tokens processed. Both the tokens sent to a model and the tokens generated are counted. Some features are billed separately.
- What are Knowledge Bases and Agents in Bedrock?
- A Knowledge Base connects a model to a set of documents so that answers can be grounded in that data. An Agent allows a model to call tools and take actions to complete a task.
This article is the summary. The book is the full, continuously updated reference: Amazon Bedrock, Knowledge Bases, Agents, AgentCore, and multi-agent systems on AWS.
View the book