feat(cloudfront): serve POST, PUT, PATCH and DELETE viewer requests (#4229)

The serving controller only declared GET, HEAD and OPTIONS, so a write
method on a distribution host matched the resource class but no method
and fell through to a bare 405, even when the cache behavior allowed all
seven methods.

POST, PUT, PATCH and DELETE are now served under the same AllowedMethods
check (403 "Invalid method." otherwise) and the same trusted key group
enforcement as the read methods. Custom origins receive the method, the
viewer request body, its Content-Type and its Authorization header, with
Content-Length taken from the body; GET and HEAD still reach the origin
without Authorization, as on CloudFront. In-process S3 origins answer
these methods with 403 Access Denied instead of forwarding them, which
is documented.

Closes #4216

(cherry picked from commit 4ca0e085cd9064cf7672cb050049e9564b48ed55)
This commit is contained in:
jwoa95
2026-09-23 17:24:40 +01:00
committed by Zakaria
parent e557be1bc6
commit aef858a47c
6 changed files with 531 additions and 17 deletions
+17 -5
View File
@@ -3,7 +3,8 @@
CloudFront management-plane and local content-delivery emulation. Supports distribution lifecycle, CloudFront management-plane and local content-delivery emulation. Supports distribution lifecycle,
cache policies, origin request policies, response headers policies, origin access controls, origin cache policies, origin request policies, response headers policies, origin access controls, origin
access identities, public keys, trusted key groups, CloudFront Functions, invalidations, tagging, and access identities, public keys, trusted key groups, CloudFront Functions, invalidations, tagging, and
GET/HEAD/OPTIONS delivery from S3 or custom origins. viewer delivery from S3 or custom origins for every CloudFront viewer method (GET, HEAD, OPTIONS, POST,
PUT, PATCH and DELETE).
**Protocol:** REST XML **Protocol:** REST XML
**API version:** `2020-05-31` **API version:** `2020-05-31`
@@ -150,9 +151,17 @@ GET/HEAD/OPTIONS delivery from S3 or custom origins.
- CNAME aliases are globally unique. `AssociateAlias` atomically transfers an alias from its current - CNAME aliases are globally unique. `AssociateAlias` atomically transfers an alias from its current
owner to the target distribution. Exact aliases take precedence over the most-specific matching owner to the target distribution. Exact aliases take precedence over the most-specific matching
wildcard alias. wildcard alias.
- Viewer GET/HEAD requests, and OPTIONS requests allowed by the matched cache behavior, addressed to - Viewer requests addressed to an enabled distribution's generated domain or alias are routed to the
an enabled distribution's generated domain or alias are routed to the matching S3 or custom matching S3 or custom origin when the matched cache behavior's `AllowedMethods` includes the
origin. Origin forwarding preserves the raw path; custom-origin redirects are not followed. method; any other method returns 403 `Invalid method.`. A behavior without `AllowedMethods` allows
GET and HEAD. Origin forwarding preserves the raw path; custom-origin redirects are not followed.
- POST, PUT, PATCH and DELETE are forwarded to custom origins with the viewer request body and its
`Content-Type`; `Content-Length` is set from the body. Their responses are never cached, as on AWS.
Signed URL and signed cookie enforcement applies to them like any other method. Request bodies are
bounded by the emulator-wide `floci.protocols.max-request-size` limit.
- The viewer's `Authorization` header is forwarded to custom origins on POST, PUT, PATCH and DELETE
and removed from GET and HEAD requests, as on AWS. Floci also removes it from OPTIONS requests;
AWS removes it there only when OPTIONS is a cached method.
- Every distribution is also served as `{id}.cloudfront.{host}` for each endpoint host Floci - Every distribution is also served as `{id}.cloudfront.{host}` for each endpoint host Floci
resolves: `localhost`, `localhost.floci.io`, `localhost.localstack.cloud`, `FLOCI_HOSTNAME` and resolves: `localhost`, `localhost.floci.io`, `localhost.localstack.cloud`, `FLOCI_HOSTNAME` and
every `FLOCI_DNS_EXTRA_SUFFIXES` entry. `{id}.cloudfront.localhost.floci.io` and every `FLOCI_DNS_EXTRA_SUFFIXES` entry. `{id}.cloudfront.localhost.floci.io` and
@@ -160,7 +169,7 @@ GET/HEAD/OPTIONS delivery from S3 or custom origins.
HTTPS certificate, so a signed URL for either can be downloaded over `https://`. See HTTPS certificate, so a signed URL for either can be downloaded over `https://`. See
[Downloading over HTTPS](#downloading-over-https). [Downloading over HTTPS](#downloading-over-https).
- Origin custom headers are persisted through the CloudFront API and CloudFormation. They replace - Origin custom headers are persisted through the CloudFront API and CloudFormation. They replace
same-named viewer headers on custom-origin GET/HEAD/OPTIONS requests. For in-process S3 origins, a same-named viewer headers on every custom-origin request. For in-process S3 origins, a
configured `Origin` header is used for S3 CORS evaluation. AWS-prohibited names, malformed configured `Origin` header is used for S3 CORS evaluation. AWS-prohibited names, malformed
values, inconsistent quantities, duplicates, and quota violations are rejected with modeled values, inconsistent quantities, duplicates, and quota violations are rejected with modeled
CloudFront errors when the distribution is created or updated. CloudFront errors when the distribution is created or updated.
@@ -339,3 +348,6 @@ as it is.
- CloudFormation provisioning of custom `AWS::CloudFront::ResponseHeadersPolicy` resources - CloudFormation provisioning of custom `AWS::CloudFront::ResponseHeadersPolicy` resources
(literal custom or managed policy IDs are supported on distributions) (literal custom or managed policy IDs are supported on distributions)
- Persistent edge caching and global CDN propagation - Persistent edge caching and global CDN propagation
- Forwarding POST, PUT, PATCH and DELETE to S3 origins. AWS passes them to S3, which evaluates them
against the bucket policy (OAC supports `PUT` and `DELETE`); Floci's in-process S3 origin answers
them with 403 `Access Denied`, as S3 does for a request that holds no write grant.
@@ -16,6 +16,7 @@ import org.apache.hc.client5.http.protocol.HttpClientContext;
import org.apache.hc.client5.http.ssl.ClientTlsStrategyBuilder; import org.apache.hc.client5.http.ssl.ClientTlsStrategyBuilder;
import org.apache.hc.core5.http.Header; import org.apache.hc.core5.http.Header;
import org.apache.hc.core5.http.HttpVersion; import org.apache.hc.core5.http.HttpVersion;
import org.apache.hc.core5.http.io.entity.ByteArrayEntity;
import org.apache.hc.core5.http.io.entity.EntityUtils; import org.apache.hc.core5.http.io.entity.EntityUtils;
import org.apache.hc.core5.http.io.support.ClassicRequestBuilder; import org.apache.hc.core5.http.io.support.ClassicRequestBuilder;
import org.apache.hc.core5.http2.HttpVersionPolicy; import org.apache.hc.core5.http2.HttpVersionPolicy;
@@ -119,11 +120,22 @@ final class CloudFrontOriginHttpClient implements AutoCloseable {
HttpResponse<byte[]> send(HttpRequest request, Map<String, String> originHeaders, HttpResponse<byte[]> send(HttpRequest request, Map<String, String> originHeaders,
HttpResponse.BodyHandler<byte[]> bodyHandler) HttpResponse.BodyHandler<byte[]> bodyHandler)
throws IOException, InterruptedException { throws IOException, InterruptedException {
return send(request, originHeaders, null, bodyHandler);
}
/**
* Sends {@code request} with {@code requestBody} as its entity, or with no entity when it is
* {@code null}. The request's own body publisher must be empty: the body travels as bytes so its
* {@code Content-Length} is exact.
*/
HttpResponse<byte[]> send(HttpRequest request, Map<String, String> originHeaders,
byte[] requestBody, HttpResponse.BodyHandler<byte[]> bodyHandler)
throws IOException, InterruptedException {
if (Thread.currentThread().isInterrupted()) { if (Thread.currentThread().isInterrupted()) {
throw new InterruptedException("CloudFront origin request interrupted"); throw new InterruptedException("CloudFront origin request interrupted");
} }
if (request.bodyPublisher().filter(publisher -> publisher.contentLength() != 0).isPresent()) { if (request.bodyPublisher().filter(publisher -> publisher.contentLength() != 0).isPresent()) {
throw new IOException("CloudFront origin request bodies are not supported"); throw new IOException("CloudFront origin request bodies must be passed as bytes");
} }
ClassicRequestBuilder builder = ClassicRequestBuilder.create(request.method()) ClassicRequestBuilder builder = ClassicRequestBuilder.create(request.method())
@@ -132,6 +144,10 @@ final class CloudFrontOriginHttpClient implements AutoCloseable {
request.headers().map().forEach((name, values) -> request.headers().map().forEach((name, values) ->
values.forEach(value -> builder.addHeader(name, value))); values.forEach(value -> builder.addHeader(name, value)));
originHeaders.forEach(builder::setHeader); originHeaders.forEach(builder::setHeader);
if (requestBody != null) {
// No entity content type: the caller's Content-Type request header is sent as is.
builder.setEntity(new ByteArrayEntity(requestBody, null));
}
HttpClientContext context = HttpClientContext.create(); HttpClientContext context = HttpClientContext.create();
Duration responseTimeout = request.timeout().orElse(RESPONSE_TIMEOUT); Duration responseTimeout = request.timeout().orElse(RESPONSE_TIMEOUT);
@@ -10,11 +10,17 @@ import io.github.hectorvent.floci.services.cloudfront.model.OriginAccessControl;
import io.github.hectorvent.floci.services.s3.S3Service; import io.github.hectorvent.floci.services.s3.S3Service;
import io.github.hectorvent.floci.services.s3.model.S3Object; import io.github.hectorvent.floci.services.s3.model.S3Object;
import io.quarkus.vertx.http.runtime.CurrentVertxRequest; import io.quarkus.vertx.http.runtime.CurrentVertxRequest;
import io.vertx.core.http.HttpServerRequest;
import jakarta.annotation.PreDestroy; import jakarta.annotation.PreDestroy;
import jakarta.inject.Inject; import jakarta.inject.Inject;
import jakarta.ws.rs.Consumes;
import jakarta.ws.rs.DELETE;
import jakarta.ws.rs.GET; import jakarta.ws.rs.GET;
import jakarta.ws.rs.HEAD; import jakarta.ws.rs.HEAD;
import jakarta.ws.rs.OPTIONS; import jakarta.ws.rs.OPTIONS;
import jakarta.ws.rs.PATCH;
import jakarta.ws.rs.POST;
import jakarta.ws.rs.PUT;
import jakarta.ws.rs.Path; import jakarta.ws.rs.Path;
import jakarta.ws.rs.PathParam; import jakarta.ws.rs.PathParam;
import jakarta.ws.rs.core.Context; import jakarta.ws.rs.core.Context;
@@ -60,7 +66,10 @@ import java.util.Set;
* </ul> * </ul>
* *
* <p>Viewer-protocol-policy enforcement is intentionally out of scope for this layer: the emulator * <p>Viewer-protocol-policy enforcement is intentionally out of scope for this layer: the emulator
* is HTTP-first. GET/HEAD are served, and OPTIONS is served only when the matched behavior allows it. * is HTTP-first. Every CloudFront viewer method (GET, HEAD, OPTIONS, POST, PUT, PATCH, DELETE) is
* served when the matched behavior's {@code AllowedMethods} includes it. POST, PUT, PATCH and DELETE
* are forwarded to custom origins with the viewer request body and {@code Authorization} header; they
* are not forwarded to in-process S3 origins.
*/ */
@Path("/_cloudfront/{distId}") @Path("/_cloudfront/{distId}")
public class CloudFrontServingController { public class CloudFrontServingController {
@@ -80,6 +89,8 @@ public class CloudFrontServingController {
private static final java.util.Set<String> CLOUDFRONT_SIGNING_PARAMS = private static final java.util.Set<String> CLOUDFRONT_SIGNING_PARAMS =
java.util.Set.of("Expires", "Signature", "Key-Pair-Id", "Policy", "Hash-Algorithm"); java.util.Set.of("Expires", "Signature", "Key-Pair-Id", "Policy", "Hash-Algorithm");
/** Viewer methods that can carry a request body and that CloudFront never caches. */
private static final Set<String> BODY_METHODS = Set.of("POST", "PUT", "PATCH", "DELETE");
private final CloudFrontService service; private final CloudFrontService service;
private final S3Service s3Service; private final S3Service s3Service;
@@ -117,7 +128,7 @@ public class CloudFrontServingController {
RequestHost.of(request), RequestHost.of(request),
headers.getHeaderString(HttpHeaders.AUTHORIZATION), headers.getHeaderString(HttpHeaders.AUTHORIZATION),
request.getHeader("Origin"), "GET", null, null, request.getHeader("Origin"), "GET", null, null,
request.getHeader("Pragma")); request.getHeader("Pragma"), null, null);
} }
@HEAD @HEAD
@@ -131,7 +142,7 @@ public class CloudFrontServingController {
RequestHost.of(request), RequestHost.of(request),
headers.getHeaderString(HttpHeaders.AUTHORIZATION), headers.getHeaderString(HttpHeaders.AUTHORIZATION),
request.getHeader("Origin"), "HEAD", null, null, request.getHeader("Origin"), "HEAD", null, null,
request.getHeader("Pragma")); request.getHeader("Pragma"), null, null);
} }
@OPTIONS @OPTIONS
@@ -147,7 +158,53 @@ public class CloudFrontServingController {
request.getHeader("Origin"), "OPTIONS", request.getHeader("Origin"), "OPTIONS",
request.getHeader("Access-Control-Request-Method"), request.getHeader("Access-Control-Request-Method"),
request.getHeader("Access-Control-Request-Headers"), request.getHeader("Access-Control-Request-Headers"),
request.getHeader("Pragma")); request.getHeader("Pragma"), null, null);
}
@POST
@Path("/{proxy:.*}")
@Consumes(MediaType.WILDCARD)
public Response post(@PathParam("distId") String distId, @PathParam("proxy") String proxy,
@Context HttpHeaders headers, @Context UriInfo uriInfo, byte[] body) {
return serveWithBody(distId, "POST", headers, uriInfo, body);
}
@PUT
@Path("/{proxy:.*}")
@Consumes(MediaType.WILDCARD)
public Response put(@PathParam("distId") String distId, @PathParam("proxy") String proxy,
@Context HttpHeaders headers, @Context UriInfo uriInfo, byte[] body) {
return serveWithBody(distId, "PUT", headers, uriInfo, body);
}
@PATCH
@Path("/{proxy:.*}")
@Consumes(MediaType.WILDCARD)
public Response patch(@PathParam("distId") String distId, @PathParam("proxy") String proxy,
@Context HttpHeaders headers, @Context UriInfo uriInfo, byte[] body) {
return serveWithBody(distId, "PATCH", headers, uriInfo, body);
}
@DELETE
@Path("/{proxy:.*}")
@Consumes(MediaType.WILDCARD)
public Response delete(@PathParam("distId") String distId, @PathParam("proxy") String proxy,
@Context HttpHeaders headers, @Context UriInfo uriInfo, byte[] body) {
return serveWithBody(distId, "DELETE", headers, uriInfo, body);
}
private Response serveWithBody(String distId, String method, HttpHeaders headers,
UriInfo uriInfo, byte[] body) {
HttpServerRequest request = currentVertxRequest.getCurrent().request();
String rawViewerPath = rawViewerPath(request.uri());
return serve(distId, rawViewerPath, decodedViewerPath(rawViewerPath),
uriInfo.getRequestUri().getScheme(),
RequestHost.of(request),
headers.getHeaderString(HttpHeaders.AUTHORIZATION),
request.getHeader("Origin"), method, null, null,
request.getHeader("Pragma"),
body != null ? body : new byte[0],
request.getHeader(HttpHeaders.CONTENT_TYPE));
} }
private Response serve(String distId, String rawViewerPath, String decodedViewerPath, private Response serve(String distId, String rawViewerPath, String decodedViewerPath,
@@ -156,7 +213,9 @@ public class CloudFrontServingController {
String viewerOrigin, String method, String viewerOrigin, String method,
String accessControlRequestMethod, String accessControlRequestMethod,
String accessControlRequestHeaders, String accessControlRequestHeaders,
String pragma) { String pragma,
byte[] viewerBody,
String viewerContentType) {
boolean includeBody = !"HEAD".equals(method); boolean includeBody = !"HEAD".equals(method);
boolean preflightRequest = "OPTIONS".equals(method) boolean preflightRequest = "OPTIONS".equals(method)
&& viewerOrigin != null && !viewerOrigin.isBlank() && viewerOrigin != null && !viewerOrigin.isBlank()
@@ -199,7 +258,8 @@ public class CloudFrontServingController {
OriginResponse origin = route(dist, normalized, rawViewerPath, decodedViewerPath, OriginResponse origin = route(dist, normalized, rawViewerPath, decodedViewerPath,
viewerScheme, viewerAuthorization, method, viewerOrigin, viewerScheme, viewerAuthorization, method, viewerOrigin,
accessControlRequestMethod, accessControlRequestHeaders); accessControlRequestMethod, accessControlRequestHeaders,
viewerBody, viewerContentType);
if (origin.status() >= 400) { if (origin.status() >= 400) {
Response fallback = applyCustomError( Response fallback = applyCustomError(
@@ -345,7 +405,8 @@ public class CloudFrontServingController {
String viewerScheme, String viewerAuthorization, String viewerScheme, String viewerAuthorization,
String method, String viewerOrigin, String method, String viewerOrigin,
String accessControlRequestMethod, String accessControlRequestMethod,
String accessControlRequestHeaders) { String accessControlRequestHeaders,
byte[] viewerBody, String viewerContentType) {
DistributionConfig config = distribution.getConfig(); DistributionConfig config = distribution.getConfig();
String originId = CloudFrontRequestRouter.matchTargetOriginId(config, normalized); String originId = CloudFrontRequestRouter.matchTargetOriginId(config, normalized);
Origin origin = CloudFrontRequestRouter.findOrigin(config, originId); Origin origin = CloudFrontRequestRouter.findOrigin(config, originId);
@@ -357,6 +418,14 @@ public class CloudFrontServingController {
return fetchS3Preflight(origin, viewerOrigin, accessControlRequestMethod, return fetchS3Preflight(origin, viewerOrigin, accessControlRequestMethod,
accessControlRequestHeaders); accessControlRequestHeaders);
} }
if (BODY_METHODS.contains(method)) {
// AWS forwards these methods and S3 evaluates them against the bucket policy (OAC
// supports PUT and DELETE). The in-process origin authorizes reads only, so a write is
// answered as S3 answers a request that holds no write grant.
LOG.debugv("CloudFront does not forward {0} requests to in-process S3 origin {1}",
method, origin.getId());
return OriginResponse.error(403, "Access Denied");
}
String key = CloudFrontRequestRouter.resolveOriginKey( String key = CloudFrontRequestRouter.resolveOriginKey(
origin.getOriginPath(), decodedViewerPath, config.getDefaultRootObject()); origin.getOriginPath(), decodedViewerPath, config.getDefaultRootObject());
return fetchFromS3( return fetchFromS3(
@@ -368,7 +437,8 @@ public class CloudFrontServingController {
// origin request policy, or legacy ForwardedValues configuration. Those policy // origin request policy, or legacy ForwardedValues configuration. Those policy
// semantics are not modeled in the data plane yet, so the AWS default is to omit them. // semantics are not modeled in the data plane yet, so the AWS default is to omit them.
return fetchFromCustomOrigin(origin, forwardUri, null, viewerScheme, method, return fetchFromCustomOrigin(origin, forwardUri, null, viewerScheme, method,
viewerOrigin, accessControlRequestMethod, accessControlRequestHeaders); viewerOrigin, accessControlRequestMethod, accessControlRequestHeaders,
viewerAuthorization, viewerBody, viewerContentType);
} }
private OriginResponse fetchFromS3( private OriginResponse fetchFromS3(
@@ -494,7 +564,9 @@ public class CloudFrontServingController {
private OriginResponse fetchFromCustomOrigin(Origin origin, String forwardUri, String rawQuery, private OriginResponse fetchFromCustomOrigin(Origin origin, String forwardUri, String rawQuery,
String viewerScheme, String method, String viewerOrigin, String viewerScheme, String method, String viewerOrigin,
String accessControlRequestMethod, String accessControlRequestMethod,
String accessControlRequestHeaders) { String accessControlRequestHeaders,
String viewerAuthorization,
byte[] viewerBody, String viewerContentType) {
boolean includeBody = !"HEAD".equals(method); boolean includeBody = !"HEAD".equals(method);
try { try {
URI target = buildCustomOriginUri( URI target = buildCustomOriginUri(
@@ -522,8 +594,19 @@ public class CloudFrontServingController {
addRequestHeader(rb, "Access-Control-Request-Method", accessControlRequestMethod); addRequestHeader(rb, "Access-Control-Request-Method", accessControlRequestMethod);
addRequestHeader(rb, "Access-Control-Request-Headers", accessControlRequestHeaders); addRequestHeader(rb, "Access-Control-Request-Headers", accessControlRequestHeaders);
} }
if (BODY_METHODS.contains(method)) {
// CloudFront removes Authorization from GET and HEAD requests but forwards it on
// POST, PUT, PATCH and DELETE, so the origin can authenticate the write.
addRequestHeader(rb, "Authorization", viewerAuthorization);
}
byte[] originBody = originRequestBody(method, viewerBody);
if (originBody != null) {
// CloudFront forwards the viewer's Content-Type with the body; the transport derives
// Content-Length from the body itself.
addRequestHeader(rb, "Content-Type", viewerContentType);
}
HttpResponse<byte[]> resp = httpClient.send( HttpResponse<byte[]> resp = httpClient.send(
rb.build(), originHeaders, HttpResponse.BodyHandlers.ofByteArray()); rb.build(), originHeaders, originBody, HttpResponse.BodyHandlers.ofByteArray());
String ct = resp.headers().firstValue("content-type").orElse(DEFAULT_CONTENT_TYPE); String ct = resp.headers().firstValue("content-type").orElse(DEFAULT_CONTENT_TYPE);
byte[] body = resp.body() != null ? resp.body() : new byte[0]; byte[] body = resp.body() != null ? resp.body() : new byte[0];
long contentLength = includeBody ? body.length : responseContentLength(resp); long contentLength = includeBody ? body.length : responseContentLength(resp);
@@ -542,6 +625,20 @@ public class CloudFrontServingController {
} }
} }
/**
* The body CloudFront sends to the origin: POST, PUT and PATCH always carry one (possibly empty, so
* the origin still sees {@code Content-Length: 0}); DELETE carries one only when the viewer sent it.
*/
private static byte[] originRequestBody(String method, byte[] viewerBody) {
if (viewerBody == null || !BODY_METHODS.contains(method)) {
return null;
}
if ("DELETE".equals(method) && viewerBody.length == 0) {
return null;
}
return viewerBody;
}
private static String originAccessIdentityId(Origin origin) { private static String originAccessIdentityId(Origin origin) {
Map<String, String> config = origin.getS3OriginConfig(); Map<String, String> config = origin.getS3OriginConfig();
String value = config != null ? config.get("OriginAccessIdentity") : null; String value = config != null ? config.get("OriginAccessIdentity") : null;
@@ -708,7 +805,7 @@ public class CloudFrontServingController {
String forwardUri = CloudFrontRequestRouter.resolveForwardUri(errOrigin.getOriginPath(), errNormalized, null); String forwardUri = CloudFrontRequestRouter.resolveForwardUri(errOrigin.getOriginPath(), errNormalized, null);
page = fetchFromCustomOrigin( page = fetchFromCustomOrigin(
errOrigin, forwardUri, null, viewerScheme, errOrigin, forwardUri, null, viewerScheme,
includeBody ? "GET" : "HEAD", null, null, null); includeBody ? "GET" : "HEAD", null, null, null, null, null, null);
} }
if (page.status() >= 400) { if (page.status() >= 400) {
// Custom error page unavailable → return the status received from the error-page origin // Custom error page unavailable → return the status received from the error-page origin
@@ -1,11 +1,15 @@
package io.github.hectorvent.floci.services.cloudfront; package io.github.hectorvent.floci.services.cloudfront;
import com.sun.net.httpserver.Headers;
import com.sun.net.httpserver.HttpExchange; import com.sun.net.httpserver.HttpExchange;
import com.sun.net.httpserver.HttpServer; import com.sun.net.httpserver.HttpServer;
import io.github.hectorvent.floci.services.cloudfront.model.CacheBehavior;
import io.github.hectorvent.floci.services.cloudfront.model.DefaultCacheBehavior; import io.github.hectorvent.floci.services.cloudfront.model.DefaultCacheBehavior;
import io.github.hectorvent.floci.services.cloudfront.model.Distribution; import io.github.hectorvent.floci.services.cloudfront.model.Distribution;
import io.github.hectorvent.floci.services.cloudfront.model.DistributionConfig; import io.github.hectorvent.floci.services.cloudfront.model.DistributionConfig;
import io.github.hectorvent.floci.services.cloudfront.model.KeyGroup;
import io.github.hectorvent.floci.services.cloudfront.model.Origin; import io.github.hectorvent.floci.services.cloudfront.model.Origin;
import io.github.hectorvent.floci.services.cloudfront.model.PublicKey;
import io.github.hectorvent.floci.services.cloudfront.model.ResponseHeadersPolicy; import io.github.hectorvent.floci.services.cloudfront.model.ResponseHeadersPolicy;
import io.quarkus.test.junit.QuarkusTest; import io.quarkus.test.junit.QuarkusTest;
import io.quarkus.test.junit.QuarkusTestProfile; import io.quarkus.test.junit.QuarkusTestProfile;
@@ -17,13 +21,21 @@ import org.junit.jupiter.api.Test;
import java.io.IOException; import java.io.IOException;
import java.net.InetSocketAddress; import java.net.InetSocketAddress;
import java.nio.charset.StandardCharsets; import java.nio.charset.StandardCharsets;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.Signature;
import java.time.Instant;
import java.util.Base64;
import java.util.LinkedHashMap; import java.util.LinkedHashMap;
import java.util.List; import java.util.List;
import java.util.Map; import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.atomic.AtomicReference; import java.util.concurrent.atomic.AtomicReference;
import static io.restassured.RestAssured.given; import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.equalTo;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertNotEquals; import static org.junit.jupiter.api.Assertions.assertNotEquals;
@@ -220,6 +232,315 @@ class CloudFrontCustomOriginServingTest {
removeOnlyResponse.getHeader("Date")); removeOnlyResponse.getHeader("Date"));
} }
@Test
void forwardsWriteMethodsWithTheirBodyToTheOrigin() throws Exception {
AtomicReference<ReceivedRequest> received = new AtomicReference<>();
AtomicInteger hits = new AtomicInteger();
startRecordingOrigin(received, hits);
Distribution created = cloudFrontService.createDistribution(
customOriginDistribution(allMethodsBehavior()), Map.of());
String json = "{\"name\":\"widget\",\"note\":\"café\"}";
given()
.header("Host", created.getDomainName())
.contentType("application/json; charset=UTF-8")
.body(json.getBytes(StandardCharsets.UTF_8))
.when()
.post("/api/things")
.then()
.statusCode(201)
.header("X-Origin-Method", equalTo("POST"))
.body(equalTo("created"));
ReceivedRequest post = received.get();
assertEquals("POST", post.method());
assertEquals("/api/things", post.path());
assertEquals("application/json; charset=UTF-8", post.header("Content-Type"));
assertEquals(Integer.toString(json.getBytes(StandardCharsets.UTF_8).length),
post.header("Content-Length"));
assertArrayEquals(json.getBytes(StandardCharsets.UTF_8), post.body());
byte[] binary = new byte[] {0, 1, 2, (byte) 0xff, (byte) 0xfe, 10, 13};
given()
.header("Host", created.getDomainName())
.contentType("application/octet-stream")
.body(binary)
.when()
.put("/api/things/1")
.then()
.statusCode(200)
.header("X-Origin-Method", equalTo("PUT"));
assertEquals("PUT", received.get().method());
assertTrue(received.get().header("Content-Type").startsWith("application/octet-stream"));
assertArrayEquals(binary, received.get().body());
given()
.header("Host", created.getDomainName())
.contentType("application/x-www-form-urlencoded")
.body("name=widget&size=2")
.when()
.patch("/api/things/1")
.then()
.statusCode(200)
.header("X-Origin-Method", equalTo("PATCH"));
assertEquals("PATCH", received.get().method());
assertTrue(received.get().header("Content-Type")
.startsWith("application/x-www-form-urlencoded"));
assertEquals("name=widget&size=2",
new String(received.get().body(), StandardCharsets.UTF_8));
given()
.header("Host", created.getDomainName())
.when()
.delete("/api/things/1")
.then()
.statusCode(200)
.header("X-Origin-Method", equalTo("DELETE"));
assertEquals("DELETE", received.get().method());
assertEquals("/api/things/1", received.get().path());
assertEquals(0, received.get().body().length);
assertNull(received.get().header("Content-Type"));
given()
.header("Host", created.getDomainName())
.when()
.post("/api/empty")
.then()
.statusCode(201);
assertEquals("0", received.get().header("Content-Length"));
assertEquals(5, hits.get());
}
@Test
void forwardsTheViewerAuthorizationOnWriteMethodsOnly() throws Exception {
AtomicReference<ReceivedRequest> received = new AtomicReference<>();
AtomicInteger hits = new AtomicInteger();
startRecordingOrigin(received, hits);
Distribution created = cloudFrontService.createDistribution(
customOriginDistribution(allMethodsBehavior()), Map.of());
for (String method : List.of("POST", "PUT", "PATCH", "DELETE")) {
given()
.header("Host", created.getDomainName())
.header("Authorization", "Bearer viewer-token")
.when()
.request(method, "/api/things/1")
.then()
.header("X-Origin-Method", equalTo(method));
assertEquals("Bearer viewer-token", received.get().header("Authorization"), method);
}
// CloudFront removes Authorization from GET and HEAD requests.
given()
.header("Host", created.getDomainName())
.header("Authorization", "Bearer viewer-token")
.when()
.get("/api/things/1")
.then()
.statusCode(200)
.header("X-Origin-Method", equalTo("GET"));
assertNull(received.get().header("Authorization"));
assertEquals(5, hits.get());
}
@Test
void customErrorPageForAWriteIsFetchedWithoutTheViewerAuthorization() throws Exception {
Map<String, String> authorizationByRequest = new ConcurrentHashMap<>();
originServer = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
originServer.createContext("/", exchange -> {
String request = exchange.getRequestMethod() + " "
+ exchange.getRequestURI().getRawPath();
authorizationByRequest.put(request,
String.valueOf(exchange.getRequestHeaders().getFirst("Authorization")));
boolean errorPage = "GET /errors/500.html".equals(request);
byte[] body = (errorPage ? "error-page" : "failed").getBytes(StandardCharsets.UTF_8);
exchange.getResponseHeaders().add("Content-Type", "text/plain");
exchange.sendResponseHeaders(errorPage ? 200 : 500, body.length);
exchange.getResponseBody().write(body);
exchange.close();
});
originServer.start();
Map<String, Object> errorPage = new LinkedHashMap<>(Map.of(
"ErrorCode", "500", "ResponseCode", "503", "ResponsePagePath", "/errors/500.html"));
Distribution distribution = customOriginDistribution(allMethodsBehavior());
distribution.getConfig().setCustomErrorResponses(List.of(errorPage));
Distribution created = cloudFrontService.createDistribution(distribution, Map.of());
given()
.header("Host", created.getDomainName())
.header("Authorization", "Bearer viewer-token")
.contentType("application/json")
.body("{}")
.when()
.post("/api/things")
.then()
.statusCode(503)
.body(equalTo("error-page"));
assertEquals(Map.of(
"POST /api/things", "Bearer viewer-token",
"GET /errors/500.html", "null"), authorizationByRequest);
}
@Test
void rejectsMethodsTheMatchedBehaviorDoesNotAllowWithoutContactingTheOrigin() throws Exception {
AtomicReference<ReceivedRequest> received = new AtomicReference<>();
AtomicInteger hits = new AtomicInteger();
startRecordingOrigin(received, hits);
CacheBehavior readOnly = new CacheBehavior();
readOnly.setPathPattern("/static/*");
readOnly.setTargetOriginId("custom-origin");
readOnly.setViewerProtocolPolicy("allow-all");
readOnly.setAllowedMethods(List.of("GET", "HEAD"));
DistributionConfig config = customOriginDistribution(allMethodsBehavior()).getConfig();
config.setCacheBehaviors(List.of(readOnly));
Distribution distribution = new Distribution();
distribution.setConfig(config);
Distribution created = cloudFrontService.createDistribution(distribution, Map.of());
for (String method : List.of("POST", "PUT", "PATCH", "DELETE")) {
given()
.header("Host", created.getDomainName())
.body("ignored")
.when()
.request(method, "/static/app.js")
.then()
.statusCode(403)
.body(equalTo("Invalid method."));
}
assertEquals(0, hits.get());
// A behavior without AllowedMethods keeps CloudFront's GET/HEAD minimum.
Distribution defaultsOnly = cloudFrontService.createDistribution(
customOriginDistribution(defaultBehavior("custom-origin")), Map.of());
given()
.header("Host", defaultsOnly.getDomainName())
.body("ignored")
.when()
.post("/api/things")
.then()
.statusCode(403)
.body(equalTo("Invalid method."));
assertEquals(0, hits.get());
}
@Test
void signedBehaviorsRequireASignatureForWriteMethods() throws Exception {
AtomicReference<ReceivedRequest> received = new AtomicReference<>();
AtomicInteger hits = new AtomicInteger();
startRecordingOrigin(received, hits);
String suffix = Long.toString(System.nanoTime(), 36);
KeyPairGenerator generator = KeyPairGenerator.getInstance("RSA");
generator.initialize(2048);
KeyPair keyPair = generator.generateKeyPair();
PublicKey publicKey = new PublicKey();
publicKey.setName("pk-" + suffix);
publicKey.setCallerReference("cr-" + suffix);
publicKey.setEncodedKey("-----BEGIN PUBLIC KEY-----\n"
+ Base64.getMimeEncoder().encodeToString(keyPair.getPublic().getEncoded())
+ "\n-----END PUBLIC KEY-----");
publicKey = cloudFrontService.createPublicKey(publicKey);
KeyGroup keyGroup = new KeyGroup();
keyGroup.setName("kg-" + suffix);
keyGroup.setItems(List.of(publicKey.getId()));
keyGroup = cloudFrontService.createKeyGroup(keyGroup);
DefaultCacheBehavior behavior = allMethodsBehavior();
behavior.setTrustedKeyGroups(List.of(keyGroup.getId()));
Distribution created = cloudFrontService.createDistribution(
customOriginDistribution(behavior), Map.of());
given()
.header("Host", created.getDomainName())
.contentType("application/json")
.body("{\"unsigned\":true}")
.when()
.post("/api/things")
.then()
.statusCode(403);
assertEquals(0, hits.get());
long expires = Instant.now().getEpochSecond() + 3600;
String policy = "{\"Statement\":[{\"Resource\":\"*\",\"Condition\":{\"DateLessThan\":"
+ "{\"AWS:EpochTime\":" + expires + "}}}]}";
Signature signer = Signature.getInstance("SHA1withRSA");
signer.initSign(keyPair.getPrivate());
signer.update(policy.getBytes(StandardCharsets.UTF_8));
given()
.header("Host", created.getDomainName())
.queryParam("Policy", cfBase64(policy.getBytes(StandardCharsets.UTF_8)))
.queryParam("Signature", cfBase64(signer.sign()))
.queryParam("Key-Pair-Id", publicKey.getId())
.contentType("application/json")
.body("{\"signed\":true}")
.when()
.post("/api/things")
.then()
.statusCode(201);
assertEquals(1, hits.get());
assertEquals("{\"signed\":true}",
new String(received.get().body(), StandardCharsets.UTF_8));
assertNull(received.get().query());
}
private void startRecordingOrigin(AtomicReference<ReceivedRequest> received, AtomicInteger hits)
throws IOException {
originServer = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
originServer.createContext("/", exchange -> {
hits.incrementAndGet();
received.set(new ReceivedRequest(
exchange.getRequestMethod(),
exchange.getRequestURI().getRawPath(),
exchange.getRequestURI().getRawQuery(),
exchange.getRequestHeaders(),
exchange.getRequestBody().readAllBytes()));
byte[] body = ("POST".equals(exchange.getRequestMethod()) ? "created" : "ok")
.getBytes(StandardCharsets.UTF_8);
exchange.getResponseHeaders().add("Content-Type", "text/plain");
exchange.getResponseHeaders().add("X-Origin-Method", exchange.getRequestMethod());
exchange.sendResponseHeaders(
"POST".equals(exchange.getRequestMethod()) ? 201 : 200, body.length);
exchange.getResponseBody().write(body);
exchange.close();
});
originServer.start();
}
private Distribution customOriginDistribution(DefaultCacheBehavior behavior) {
Origin customOrigin = new Origin();
customOrigin.setId("custom-origin");
customOrigin.setDomainName("127.0.0.1");
customOrigin.setCustomOriginConfig(customOriginConfig(originServer.getAddress().getPort()));
DistributionConfig config = new DistributionConfig();
config.setEnabled(true);
config.setOrigins(List.of(customOrigin));
config.setDefaultCacheBehavior(behavior);
Distribution distribution = new Distribution();
distribution.setConfig(config);
return distribution;
}
private static DefaultCacheBehavior allMethodsBehavior() {
DefaultCacheBehavior behavior = defaultBehavior("custom-origin");
behavior.setAllowedMethods(List.of("GET", "HEAD", "OPTIONS", "PUT", "POST", "PATCH", "DELETE"));
behavior.setCachedMethods(List.of("GET", "HEAD"));
return behavior;
}
private static String cfBase64(byte[] bytes) {
return Base64.getEncoder().encodeToString(bytes)
.replace('+', '-').replace('=', '_').replace('/', '~');
}
private record ReceivedRequest(String method, String path, String query, Headers headers,
byte[] body) {
String header(String name) {
return headers.getFirst(name);
}
}
private static void respond(HttpExchange exchange, AtomicReference<String> receivedQuery, private static void respond(HttpExchange exchange, AtomicReference<String> receivedQuery,
AtomicReference<String> receivedPath, AtomicReference<String> receivedPath,
AtomicReference<String> receivedCustomHeader, AtomicReference<String> receivedCustomHeader,
@@ -1424,6 +1424,34 @@ class CloudFrontDistributionServingTest {
.then().statusCode(200); .then().statusCode(200);
} }
@Test
void writeMethodsAreNotForwardedToInProcessS3Origins() {
String suffix = suffix();
String bucket = "cf-writes-" + suffix;
createBucket(bucket);
putObject(bucket, "doc.txt", "ORIGINAL-" + suffix, "text/plain");
DefaultCacheBehavior dflt = defaultBehavior("only-origin");
dflt.setAllowedMethods(List.of("GET", "HEAD", "OPTIONS", "PUT", "POST", "PATCH", "DELETE"));
DistributionConfig config = new DistributionConfig();
config.setEnabled(true);
config.setOrigins(List.of(s3Origin("only-origin", bucket)));
config.setDefaultCacheBehavior(dflt);
Distribution distribution = cloudFrontService.createDistribution(distribution(config), Map.of());
for (String method : List.of("POST", "PUT", "PATCH", "DELETE")) {
given()
.header("Host", distribution.getDomainName())
.body("REPLACED-" + suffix)
.when().request(method, "/doc.txt")
.then().statusCode(403)
.body(equalTo("Access Denied"));
}
given().header("Host", distribution.getDomainName()).when().get("/doc.txt")
.then().statusCode(200).body(equalTo("ORIGINAL-" + suffix));
}
@Test @Test
void managedPreflightPoliciesEmitOnlyAllowOriginForSimpleRequests() { void managedPreflightPoliciesEmitOnlyAllowOriginForSimpleRequests() {
String suffix = suffix(); String suffix = suffix();
@@ -173,6 +173,46 @@ class CloudFrontOriginHttpClientTest {
} }
} }
@Test
void sendsTheBodyWithItsExactLengthAndTheCallerContentType() throws Exception {
AtomicReference<String> method = new AtomicReference<>();
AtomicReference<String> contentType = new AtomicReference<>();
AtomicReference<String> contentLength = new AtomicReference<>();
AtomicReference<byte[]> body = new AtomicReference<>();
HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
server.createContext("/", exchange -> {
method.set(exchange.getRequestMethod());
contentType.set(exchange.getRequestHeaders().getFirst("Content-Type"));
contentLength.set(exchange.getRequestHeaders().getFirst("Content-Length"));
body.set(exchange.getRequestBody().readAllBytes());
exchange.sendResponseHeaders(204, -1);
exchange.close();
});
server.start();
byte[] payload = "{\"a\":1}".getBytes(StandardCharsets.UTF_8);
try (CloudFrontOriginHttpClient client = new CloudFrontOriginHttpClient(
resolver(InetAddress.getByName("127.0.0.1")), List.of("origin.invalid"))) {
HttpRequest request = HttpRequest.newBuilder(java.net.URI.create(
"http://origin.invalid:" + server.getAddress().getPort() + "/items"))
.timeout(Duration.ofSeconds(5))
.header("Content-Type", "application/json")
.method("PUT", HttpRequest.BodyPublishers.noBody())
.build();
HttpResponse<byte[]> response = client.send(
request, Map.of(), payload, HttpResponse.BodyHandlers.ofByteArray());
assertEquals(204, response.statusCode());
assertEquals("PUT", method.get());
assertEquals("application/json", contentType.get());
assertEquals(Integer.toString(payload.length), contentLength.get());
assertArrayEquals(payload, body.get());
} finally {
server.stop(0);
}
}
@Test @Test
void httpsPinningPreservesSniAndHostnameVerification() throws Exception { void httpsPinningPreservesSniAndHostnameVerification() throws Exception {
if (Security.getProvider(BouncyCastleProvider.PROVIDER_NAME) == null) { if (Security.getProvider(BouncyCastleProvider.PROVIDER_NAME) == null) {