After reading this blog post you will be able to connect your Salesforce org to the AgentQL API and process the extracted data, provided by their crawler.

What is AgentQL and why should I care?

AgentQL ‘connects LLMs and AI agents to the entire web’ and makes it a web crawler on steroids. Given an URL or file as an input it will crawl the contents of said source. Now web crawlers have existed long before AI was the big new buzz but the combination of both makes this one here pretty interesting.

Ok, so what’s a real life use case?

Let’s say you have a data quality team that inserts new leads into your system based on web research. They will spend quite some time copy+pasting the information they gathered on the web into your org. What if I told you all you need to do is enter the relevant website and AgentQL will query all contact information and with the integration I am about to show, you will automatically create a new lead with all relevant info?

So if you are into AI, Webcrawling and Automation – here we go:

To make this happen we are going to create:

1. a named credential and an external credential
2. an apex class that does the callout aka sending the inputs to the AgentQL API and receiving the response
3. a flow to trigger the apex class and (optionally) retrieve the data
4. a prompt in the AgentQL playground to use later for our callout
5. custom metadata to store the prompt in salesforce and make it more available outside the code

Here is a high level overview of the process:

• Screen flow will take an url as an input that then
• triggers an apex class that will do the callout by calling the named credential
• the callout consists of sending the url and a pre-defined prompt from the custom metadata on what to crawl to the endpoint of AgentQL
• AgentQL will crawl the send url and return the response as a json
• the apex class will process the response and use it to create a new lead based on the mapping we provide

Step 1: Creating a named and external credential

Before we can even make the callout to AgentQL we have to take care of authentication – from my experience the most annoying part of every integration. So how are we tackling this here?

As annoying as it is we have to setup an external credential before we can create a named credential as you will get asked to chose one when creating the named credential.

Setup → Named Credential → External Credential → New
(for whatever reason you cannot search for the external credential directly in setup…)

For ‘authentication protocol’ we chose custom.

We create a new ‘Principal’. This is necessary to give access permissions to you users. The then created external credential will be available to set inside a permission set:

Full disclosure: I just copy+paste random values for the Name and Value for the Paramater 1 because they are never asked for again. Just set something…

Create a new permission set:

Go for the ‘External Credential Principal Access’ and hit ‘Edit’ on the next page:

Chose and add the Prinicipal you created in the step above:

Assign the permission set to your user and whoever should be able to use the flow / do the callout later.

Coming back to the setup and finally creating the named credential:

The URL is: ‘https://api.agentql.com/v1/query-data‘ (source)

Time to finalize our Step 1: Adding the free API Key from AgentQL as a custom header.

Where do i get the Key? https://dev.agentql.com/api-keys

(you have to register – key will be for free for the first 300 Callouts – paid tiers are also available – https://www.agentql.com/pricing)

After receiving the key you will have to create a customer header in the named credential where you will enter it like this:

Name: X-API-KEY

Value: the API Key provided by AgentQL (no, that’s not my key you see there)

Step 1 was all about authenticating with AgentQL. Now lets mover over to Step 2 which is creating a prompt in the AgentQL Playground and storing it in a custom metadata type in salesforce.

https://dev.agentql.com/playground

Besides entering a URL or uploading a file AgentQL expects to give you a proper structured prompt so it knows what to query. You can use ‘Suggest a query’ and write in human language what data you are hoping to retrieve. (Fun fact: you can also upload a file to your GoogleDrive or S3 Bucket and use that link as an URL)

As it is called playground for a reason you can play around and see which prompt fits best to your use case. Remember, in our case we are looking for company information as we want to create new leads. (you can find my final prompt in the documentation: https://salesforcepanda.com/agentql-connection/)

Once you have your prompt, you can use ‘Fetch Data’ to see what comes back:

If you are happy with your prompt you can put into a custom metadata type in salesforce. Why? Because you don’t want to change your apex class everytime you change your prompt! The custom metadata will be queried in the code and can be changed at any point via the UI.

Creating custom metadata types:

Setup → Custom Metadata Types → New → Create a new field

My custom metadata type is called ‘AgentQLQuery’ and my field is ‘called Query__c’ and I made it a ‘Long Text Area’ because some URLs might be longer than 255 characters.

Next hit ‘Manage AgentQL Query’ and then ‘New’. Here you can then give it a name and enter the actual query you created in the AgentQL Playground.

When you want to change your query can can just try something new in the playground and then do the changes here.

Step 3: The apex class for our callout

Here is a linkout the the apex class documentation to not make this blog post explode: https://salesforcepanda.com/agentql-connection/

What the apex class does:

• query the custom metadata to get the prompt
• Initiate the callout using our named credential
  • sending the url from the flow (will get to it in a second) and the prompt to https://api.agentql.com/v1/query-data
• Retrieve the response and map the data into lead fields
• create a new lead

Step 4: Creating a screenflow to bring it all together

Now that we have the prompt and the callout, we need to make it accessable to the user. How can he call the class? Where does he see the results?

I created a super simple screen flow that takes a ‘Long Text Area’ as an input. I called that field ‘url’

To get the Apex Call I use the ‘Action’ Element and search for ‘Call AgentQLÄ. The reason the class shows up in the flow at all and under this name is because inside our apex class we declared:

@InvocableMethod(label='Call Agent QL' description='Fetches data from AgentQL API')Code language: JavaScript (javascript)

Activate that ‘url’ can be set as an input and populate it with your field from the screen flow. ‘url’ shows up here because we declared in our apex class:

public class RequestWrapper {
   @InvocableVariable(label='url' description='URL to be crawled')
   public String url;
}Code language: PHP (php)

Finally we put the screen flow into our utility app in the App Manager

Setup → App Manager → chose your lightning app → Utility Items → Add flow

And voilá:

What is now the smart thing to do?

Just review of what we are able to achieve:

• Connect Salesforce to the AgentQL API
• Write whatever prompt for whatever website we want
• Retrieve the data and map it to whatever object in salesforce

The question is: What’s not possible now?

Ressources:

• https://www.agentql.com/
• https://dev.agentql.com/playground
• https://docs.agentql.com/rest-api/api-reference
• https://docs.agentql.com/accuracy/contextual-queries
• https://dev.agentql.com/api-keys
• https://www.agentql.com/pricing

Documentation: https://salesforcepanda.com/agentql-connection/