Requests

Body

The body of a Request or Response.

Calling to<FORMAT> will try to convert the body to the desired format.

Constructors

Constructor

new Body(data: string | number[] | Uint8Array): Body

Parameters

Parameter	Type
data	string | number[] | Uint8Array

Returns

Body

Properties

length

readonly length: number

The length of the body in bytes.

Methods

toJson()

toJson(): unknown

Try to parse the body as JSON.

Returns

unknown

Throws

If the body is not valid JSON.

toRaw()

toRaw(): Uint8Array

Get the raw body as an array of bytes.

Returns

Uint8Array

toText()

toText(): string

Parse the body as a string.

Unprintable characters will be replaced with �.

Returns

string

---

RequestSpec

A mutable Request that has not yet been sent.

Constructors

Constructor

new RequestSpec(url: string): RequestSpec

Build a new RequestSpec from a URL string. We try to infer as much information as possible from the URL, including the scheme, host, path and query.

You can convert a saved immutable Request object into a RequestSpec object by using the toSpec() method.

By default:

• Method is GET.
• Path is /.

Parameters

Parameter	Type
url	string

Returns

RequestSpec

Throws

If the URL is invalid.

Example

const spec = new RequestSpec("https://example.com");

Methods

getBody()

getBody(): Body | undefined

The body of the request.

Returns

Body | undefined

getHeader()

getHeader(name: string | GetHeaderOptions): string[] | undefined

Get a header value.

Header name is case-insensitive. The header might have multiple values.

Parameters

Parameter	Type
name	string | GetHeaderOptions

Returns

string[] | undefined

getHeaders()

getHeaders(): Record<string, string[]>

The headers of the request.

Header names are case-insensitive. Each header might have multiple values.

Returns

Record<string, string[]>

Example

{
  "Host": ["caido.io"],
  "Connection": ["keep-alive"],
  "Content-Length": ["95"]
}

getHost()

getHost(): string

Get the host of the request.

Returns

string

getInfo()

getInfo(): ConnectionInfo

Get the connection information of the request.

Returns

ConnectionInfo

The connection information.

getMethod()

Call Signature

getMethod(): string

Get the HTTP method of the request.

Get the raw version by passing { raw: true } in the options.

Returns

string

Call Signature

getMethod(options: RawOption): Uint8Array

Get the HTTP method of the request.

Get the raw version by passing { raw: true } in the options.

Parameters

Parameter	Type
options	RawOption

Returns

Uint8Array

getPath()

Call Signature

getPath(): string

Get the path of the request.

Get the raw version by passing { raw: true } in the options.

Returns

string

Call Signature

getPath(options: RawOption): Uint8Array

Get the path of the request.

Get the raw version by passing { raw: true } in the options.

Parameters

Parameter	Type
options	RawOption

Returns

Uint8Array

getPort()

getPort(): number

Get the port of the request.

Returns

number

getQuery()

Call Signature

getQuery(): string

Get the unparsed query of the request.

Get the raw version by passing { raw: true } in the options.

Excludes the leading ?.

Returns

string

Call Signature

getQuery(options: RawOption): Uint8Array

Get the unparsed query of the request.

Get the raw version by passing { raw: true } in the options.

Excludes the leading ?.

Parameters

Parameter	Type
options	RawOption

Returns

Uint8Array

getRaw()

getRaw(): RequestSpecRaw

This methods converts the RequestSpec to a RequestSpecRaw.

This is useful to retrieve the raw bytes of the request.

Returns

RequestSpecRaw

Example

const spec = new RequestSpec("https://example.com");
const specRaw = spec.getRaw();
const bytes = specRaw.getRaw(); // GET / HTTP/1.1\r\nHost: example.com\r\n\r\n

getTls()

getTls(): boolean

Get if the request uses TLS (HTTPS).

Returns

boolean

getUrl()

getUrl(): string

The full URL of the request.

Returns

string

removeHeader()

removeHeader(name: string): void

Removes a header.

Parameters

Parameter	Type
name	string

Returns

void

setBody()

setBody(body: Bytes | Body, options?: SetBodyOptions): void

Set the body of the request.

The body can either be a Body or any type that can be converted to Bytes.

Parameters

Parameter	Type
body	Bytes | Body
options?	SetBodyOptions

Returns

void

Example

const body = new Body("Hello world.");
const options = { updateContentLength: true };
request.setBody(body, options);

setHeader()

setHeader(name: string, value: string): void

Set a header value.

This will overwrite any existing values.

Parameters

Parameter	Type
name	string
value	string

Returns

void

setHost()

setHost(host: string): void

Set the host of the request.

It will also update the Host header.

Parameters

Parameter	Type
host	string

Returns

void

setMethod()

setMethod(method: Bytes): void

Set the HTTP method of the request.

All strings are accepted.

Parameters

Parameter	Type
method	Bytes

Returns

void

setPath()

setPath(path: Bytes): void

Set the path of the request.

Parameters

Parameter	Type
path	Bytes

Returns

void

setPort()

setPort(port: number): void

Set the port of the request.

The port number must be between 1 and 65535.

Parameters

Parameter	Type
port	number

Returns

void

setQuery()

setQuery(query: Bytes): void

Set the unparsed query of the request.

The query string should not include the leading ?.

Parameters

Parameter	Type
query	Bytes

Returns

void

Example

spec.setQuery("q=hello");

setRaw()

setRaw(raw: Bytes): RequestSpecRaw

This method sets the raw Bytes of the request and converts it to a RequestSpecRaw.

This is useful when you have a prepared RequestSpec and you just want to modify the raw data.

Parameters

Parameter	Type
raw	Bytes

Returns

RequestSpecRaw

Example

const rawBytes = []; // RAW BYTES HERE
const request = new RequestSpec("https://example.com");
const rawRequest = request.setRaw(rawBytes);

setTls()

setTls(tls: boolean): void

Set if the request uses TLS (HTTPS).

Parameters

Parameter	Type
tls	boolean

Returns

void

parse()

Call Signature

static parse(bytes: Bytes): RequestSpec

Parses raw bytes into a RequestSpec.

Parameters

Parameter	Type
bytes	Bytes

Returns

RequestSpec

Throws

If the bytes are not a valid HTTP request.

Example

const rawInput = 'GET / HTTP/1.1\r\nHost: example.com\r\n\r\n';
const spec = RequestSpec.parse(rawInput);
spec.setHeader('x-caido', 'test');
const specRaw = spec.getRaw();
const rawOutput = specRaw.getRaw(); // Will contain the new header

Call Signature

static parse(raw: RequestSpecRaw): RequestSpec

Parses the raw bytes of a RequestSpecRaw into a RequestSpec.

Parameters

Parameter	Type
raw	RequestSpecRaw

Returns

RequestSpec

Throws

If the bytes are not a valid HTTP request.

---

RequestSpecRaw

A mutable raw Request that has not yet been sent.

Constructors

Constructor

new RequestSpecRaw(url: string): RequestSpecRaw

Build a new RequestSpecRaw from a URL string. Only the host, port and scheme will be parsed.

You can convert a saved immutable Request object into a RequestSpecRaw object by using the toSpecRaw() method.

You MUST use setRaw to set the raw bytes of the request.

Parameters

Parameter	Type
url	string

Returns

RequestSpecRaw

Example

const spec = new RequestSpecRaw("https://example.com");

Methods

getHost()

getHost(): string

Get the host of the request.

Returns

string

getInfo()

getInfo(): ConnectionInfo

Get the connection information of the request.

Returns

ConnectionInfo

The connection information.

getPort()

getPort(): number

Get the port of the request.

Returns

number

getRaw()

getRaw(): Uint8Array

Get the raw bytes of the request.

Returns

Uint8Array

getSpec()

getSpec(): RequestSpec

This methods converts the RequestSpecRaw to a RequestSpec.

Returns

RequestSpec

Deprecated

Use toSpec instead.

Throws

If the bytes are not a valid HTTP request.

See

RequestSpec.parse

getTls()

getTls(): boolean

Get if the request uses TLS (HTTPS).

Returns

boolean

setHost()

setHost(host: string): void

Set the host of the request.

It will NOT update the Host header.

Parameters

Parameter	Type
host	string

Returns

void

setPort()

setPort(port: number): void

Set the port of the request.

The port number must be between 1 and 65535.

Parameters

Parameter	Type
port	number

Returns

void

setRaw()

setRaw(raw: Bytes): void

Set the raw Bytes of the request.

Parameters

Parameter	Type
raw	Bytes

Returns

void

setTls()

setTls(tls: boolean): void

Set if the request uses TLS (HTTPS).

Parameters

Parameter	Type
tls	boolean

Returns

void

toSpec()

toSpec(): RequestSpec

This methods converts the RequestSpecRaw to a RequestSpec.

Returns

RequestSpec

Throws

If the bytes are not a valid HTTP request.

See

RequestSpec.parse

---

GetHeaderOptions

GetHeaderOptions = object

Options for getting a header value.

Properties

name

name: string

The name of the header to get. Header name is case-insensitive.

split?

optional split: boolean

Whether to split the header value on commas.

Default

false

---

Request

Request = object

An immutable saved Request.

To modify, use toSpec to get a RequestSpec object.

Methods

getBody()

getBody(): Body | undefined

The body of the request.

Returns

Body | undefined

getCreatedAt()

getCreatedAt(): Date

The datetime the request was recorded by the proxy.

Returns

Date

getHeader()

getHeader(name: string | GetHeaderOptions): string[] | undefined

Get a header value.

Header name is case-insensitive. The header might have multiple values.

Parameters

Parameter	Type
name	string | GetHeaderOptions

Returns

string[] | undefined

getHeaders()

getHeaders(): Record<string, string[]>

The headers of the request.

Header names are case-insensitive. Each header might have multiple values.

Returns

Record<string, string[]>

Example

{
  "Host": ["caido.io"],
  "Connection": ["keep-alive"],
  "Content-Length": ["95"]
}

getHost()

getHost(): string

The target host of the request.

Returns

string

getId()

getId(): ID

The unique Caido ID of the request.

Returns

ID

getMethod()

getMethod(): string

The HTTP method of the request.

Returns

string

getPath()

getPath(): string

The path of the request.

Returns

string

getPort()

getPort(): number

The target port of the request.

Returns

number

getQuery()

getQuery(): string

The unparsed query of the request.

Excludes the leading ?.

Returns

string

getRaw()

getRaw(): RequestRaw

The raw version of the request.

Used to access the bytes directly.

Returns

RequestRaw

getTls()

getTls(): boolean

If the request uses TLS (HTTPS).

Returns

boolean

getUrl()

getUrl(): string

The full URL of the request.

Returns

string

toSpec()

toSpec(): RequestSpec

Copied the request to a mutable un-saved RequestSpec. This enables you to make modify a request before re-sending it.

Returns

RequestSpec

toSpecRaw()

toSpecRaw(): RequestSpecRaw

Copied the request to a mutable un-saved RequestSpecRaw. The raw requests are not parsed and can be used to send invalid HTTP Requests.

Returns

RequestSpecRaw

---

RequestOrderField

RequestOrderField = "ext" | "host" | "id" | "method" | "path" | "query" | "created_at" | "source"

Field to order requests by.

---

RequestRaw

RequestRaw = object

An immutable saved raw Request.

Methods

toBytes()

toBytes(): Uint8Array

Get the raw request as an array of bytes.

Returns

Uint8Array

toText()

toText(): string

Parse the raw request as a string.

Unprintable characters will be replaced with �.

Returns

string

---

RequestResponse

RequestResponse = object

An immutable saved Request and Response pair.

Properties

request

request: Request

response

response: Response

---

RequestResponseOpt

RequestResponseOpt = object

An immutable saved Request and optional Response pair.

Properties

request

request: Request

response?

optional response: Response

---

RequestsConnection

RequestsConnection = object

A connection of requests.

Properties

items

items: RequestsConnectionItem[]

pageInfo

pageInfo: PageInfo

---

RequestsConnectionItem

RequestsConnectionItem = object

An item in a connection of requests.

Properties

cursor

cursor: Cursor

request

request: Request

response?

optional response: Response

---

RequestSendOptions

RequestSendOptions = object

Options for sending a request.

Properties

connection?

optional connection: Connection

The Connection to use for the request.

If provided, the request will be sent through the connection.

If not provided, the engine will open a new connection to the target.

Default

undefined

plugins?

optional plugins: boolean

If true, the request will be sent through the upstream plugins.

It defaults to to true most of the time except when called from a onUpstream callback.

Default

true

save?

optional save: boolean

If true, the request and response will be saved to the database and the user will see them in the Search tab.

If you do not save, the request and response IDs will be set to 0.

Default

true

timeouts?

optional timeouts: RequestSendTimeouts | number

The timeouts to use for sending a request and receiving a response.

If a number is provided, it will be used as the global timeout and the other timeouts will be set to infinity.

See the RequestSendTimeouts for the default values.

---

RequestSendPayload

RequestSendPayload = RequestResponse & object

A saved Request and Response pair with the connection used to send the request.

Type Declaration

connection

connection: Connection

---

RequestSendTimeouts

RequestSendTimeouts = object

Timeouts for sending a request and receiving a response.

Properties

connect?

optional connect: number

The timeout to open the TCP connection to the target host and perform the TLS handshake.

Defaults to 30s.

extra?

optional extra: number

The timeout to read data after we have a read the full response.

This is useful if you believe the server will send more data than implied by the Content-Length header.

Defaults to 0s (no timeout).

global?

optional global: number

The global timeout for sending a request and receiving a response.

No default value.

partial?

optional partial: number

The timeout between each read attempt for the response. On a slow connection, this is important to increase.

Defaults to 5s.

response?

optional response: number

The timeout to receive the first byte of the response.

After the first byte is received, the partial timeout will be used.

Defaults to 30s.

---

RequestsQuery

RequestsQuery = object

Query builder to fetch requests.

Methods

after()

after(cursor: Cursor): RequestsQuery

Requests after a given cursor.

Parameters

Parameter	Type	Description
cursor	Cursor	Cursor of the request

Returns

RequestsQuery

ascending()

Call Signature

ascending(target: "req", field: RequestOrderField): RequestsQuery

Ascending ordering.

Parameters

Parameter	Type	Description
target	"req"	Target of the ordering: req or resp.
field	RequestOrderField	Field to order by.

Returns

RequestsQuery

Call Signature

ascending(target: "resp", field: ResponseOrderField): RequestsQuery

Parameters

Parameter	Type
target	"resp"
field	ResponseOrderField

Returns

RequestsQuery

before()

before(cursor: Cursor): RequestsQuery

Requests before a given cursor.

Parameters

Parameter	Type	Description
cursor	Cursor	Cursor of the request

Returns

RequestsQuery

descending()

Call Signature

descending(target: "req", field: RequestOrderField): RequestsQuery

Descending ordering.

Parameters

Parameter	Type	Description
target	"req"	Target of the ordering: req or resp.
field	RequestOrderField	Field to order by.

Returns

RequestsQuery

Call Signature

descending(target: "resp", field: ResponseOrderField): RequestsQuery

Parameters

Parameter	Type
target	"resp"
field	ResponseOrderField

Returns

RequestsQuery

execute()

execute(): Promise<RequestsConnection>

Execute the query.

Returns

Promise<RequestsConnection>

Throws

If a query parameter is invalid or the query cannot be executed.

filter()

filter(filter: string): RequestsQuery

Filter requests.

Parameters

Parameter	Type	Description
filter	string	HTTPQL filter

Returns

RequestsQuery

first()

first(n: number): RequestsQuery

First n requests.

Parameters

Parameter	Type	Description
n	number	Number of requests to return

Returns

RequestsQuery

last()

last(n: number): RequestsQuery

Last n requests.

Parameters

Parameter	Type	Description
n	number	Number of requests to return

Returns

RequestsQuery

---

RequestsSDK

RequestsSDK = object

The SDK for the Requests service.

Methods

get()

get(id: ID): Promise<RequestResponseOpt | undefined>

Get a request by its unique ID.

Parameters

Parameter	Type
id	ID

Returns

Promise<RequestResponseOpt | undefined>

Example

await sdk.requests.get("1");

inScope()

inScope(request: RequestSpec | Request, scopes?: Scope[] | ID[]): boolean

Checks if a request is in scope.

Parameters

Parameter	Type	Description
request	RequestSpec | Request	The request to check
scopes?	Scope[] | ID[]	Optional scopes or scope IDs to check against. If not provided, checks against the default scope.

Returns

boolean

True if the request is in scope

Example

// Check against default scope
if (sdk.requests.inScope(request)) {
 sdk.console.log("In scope");
}

// Check against specific scopes
const isInScope = sdk.requests.inScope(request, [scope1, scope2]);
sdk.console.log(isInScope); // true or false

matches()

matches(filter: string, request: Request, response?: Response): boolean

Checks if a request/response matches an HTTPQL filter.

Parameters

Parameter	Type	Description
filter	string	HTTPQL filter
request	Request	The Request to match against
response?	Response	The Response to match against

Returns

boolean

query()

query(): RequestsQuery

Query requests of the current project.

Returns

RequestsQuery

Example

const page = await sqk.requests.query().first(2).execute();
sdk.console.log(`ID: ${page.items[1].request.getId()}`);

send()

send(request: RequestSpec | RequestSpecRaw, options?: RequestSendOptions): Promise<RequestSendPayload>

Sends an HTTP request, either a RequestSpec or RequestSpecRaw.

This respects the upstream proxy settings.

Parameters

Parameter	Type
request	RequestSpec | RequestSpecRaw
options?	RequestSendOptions

Returns

Promise<RequestSendPayload>

Throws

If the request cannot be sent. If the request times out, the error message will contain the word "Timeout".

Example

const spec = new RequestSpec("https://example.com");
try {
  const res = await sdk.requests.send(request)
  sdk.console.log(res.request.getId());
  sdk.console.log(res.response.getCode());
} catch (err) {
  sdk.console.error(err);
}

---

Response

Response = object

An immutable saved Response.

Methods

getBody()

getBody(): Body | undefined

The body of the response

Returns

Body | undefined

getCode()

getCode(): number

The status code of the response.

Returns

number

getCreatedAt()

getCreatedAt(): Date

The datetime the response was recorded by the proxy.

Returns

Date

getHeader()

getHeader(name: string | GetHeaderOptions): string[] | undefined

Get a header value.

Header name is case-insensitive. The header might have multiple values.

Parameters

Parameter	Type
name	string | GetHeaderOptions

Returns

string[] | undefined

getHeaders()

getHeaders(): Record<string, string[]>

The headers of the response.

Header names are case-insensitive. Each header might have multiple values.

Returns

Record<string, string[]>

Example

{
  "Date": ["Sun, 26 May 2024 10:59:21 GMT"],
  "Content-Type": ["text/html"]
}

getId()

getId(): ID

The unique Caido ID of the response.

Returns

ID

getRaw()

getRaw(): ResponseRaw

The raw version of the response.

Used to access the bytes directly.

Returns

ResponseRaw

getRoundtripTime()

getRoundtripTime(): number

The time it took to send the request and receive the response in milliseconds.

Returns

number

---

ResponseOrderField

ResponseOrderField = "length" | "roundtrip" | "code"

Field to order responses by.

---

ResponseRaw

ResponseRaw = object

An immutable saved raw Response.

Methods

toBytes()

toBytes(): Uint8Array

Get the raw response as an array of bytes.

Returns

Uint8Array

toText()

toText(): string

Parse the raw response as a string.

Unprintable characters will be replaced with �.

Returns

string

---

SetBodyOptions

SetBodyOptions = object

Options when setting the body of a Request.

Properties

updateContentLength

updateContentLength: boolean

Should update the Content-export type header.

Default

true