Voyager + AWS ELB + WebSockets w/ HTTPS


#1

I cannot figure out how to get websockets to work with Voyger and AWS. Right now my setup is such that it creates an AWS ELB using one of their certs. Works great for HTTPS traffic. When I add WebSockets to the mix it stops, This is because the ELB strips the Upgrade and Connection headers, etc. I realized that I then have to use SSl/TCP instead HTTPS/HTTP ELB setup (service.beta.kubernetes.io/aws-load-balancer-backend-protocol=tpc). But now how do I force HTTPS traffic when someone tries to get to HTTP as the X-Forwarded-Proto won’t be set by the ELB. Is there something I can do to force the traffic to HTTPS in this configuration? Is there a way to use the AWS ALB? At the very least, is there a way to get port 80 disabled on the ELB via the voyager ingress yaml? Thanks


#2

I wrote a tutorial that shows how to expose websockets using Voyager. See the details here: https://github.com/tamalsaha/voyager-grpc-ws-demo

Also, check this page: https://github.com/appscode/voyager/blob/master/docs/guides/ingress/tls/aws-cert-manager.md

Please let me know if you still have problem with the setup.


#3

Tamal, I appreciate the help. So a few things. Right now I cannot use Let’s Encrypt, so I have to terminate the SSL at the ELB using the AWS Certificate (I am trying to get our policy changed so I can use Let’s Encrypt). I cannot use the L7 configuration of the ELB because that strips the WebSocket headers. So, for the time being, I am stuck with using the AWS Cert and the AWS ELB. I cannot use the NLB because it doesn’t do SSL termination and K8s doesn’t support ALBs yet. So, three questions:

  1. Is there a way to configure the ELB to turn off port 80 and only allow 443?
  2. Is there a way in HAProxy/Voyager to know if the user went to port 443 vs port 80 on the ELB so that I can redirect them? Note that the X-Forwarded-Proto isn’t available in the header because it is a TCP ELB, not HTTPS. I am using Proxy Protocol, but that doesn’t appear to forward on the proto, just the IPs.
  3. Is there a way to have HAProxy/Voyager listen on two ports and have the ELB forward 80 to one and 443 to another? If both are clear maybe I can detect which port is used (via front end rule) and set an ACL?

Some folks suggest that you can import the Certificate from AWS, but I don’t believe you can for ones that AWS creates, just personal ones.


#4

There are actually 2 ways to get ALBs to work in K8s. CoreOS has created an ingress controller https://github.com/kubernetes-sigs/aws-alb-ingress-controller, or Zolando has one too https://github.com/zalando-incubator/kube-ingress-aws-controller. I actually use the latter in production.