Pixie is at KubeCon EU (May 16-20, 2022). Learn more🚀
Pixie is at KubeCon EU!
pixie logo
Ctrl/Cmd + K
DocsGET STARTED
Ctrl/Cmd + K

Adding End-to-End Encryption for Proxied DataPermalink

Vihang Mehta
September 21, 20212 minutes read
Principal Engineer @ New Relic, Founding Engineer @ Pixie Labs

End-to-end encryption has become increasingly popular as users demand that any data they send - a file, email, or text message - is not decipherable by any unauthorized recipients. This consumer trend is evident in the recent surge in popularity of Signal, an encrypted instant messaging service.

In this post, we’ll cover what end-to-end encryption is and walk you through how we implemented it in our system.

Why End-to-End Encryption?Permalink

Pixie is designed with a hybrid cloud architecture where data is collected and stored on the customer's environment. The cloud component is used for user management, authentication and proxying data.

This is a simplified architecture diagram of our system before end-to-end encryption.

We use standard security practices to secure data in transit; all network communication between the cluster, proxy and client is TLS encrypted.

But TLS encryption is only point-to-point. When data passes internally through our proxy, the data is temporarily unencrypted. Pixie is an open source project, so users might deploy Pixie Cloud (and the accompanying proxy) in a variety of environments. We wanted to provide privacy guarantees for users given the heterogeneity of deployment scenarios.

By adding end-to-end encryption, we can ensure that the proxy only sees an encrypted form of the telemetry data.

ImplementationPermalink

Pixie provides multiple clients for developers to interact with its platform:

  • a web UI (JavaScript)
  • a CLI (Golang)
  • APIs (client libraries: Golang, Python)

Since we needed to support E2E encryption across multiple languages, using a crypto standard with readily available implementations in multiple languages was a must. Given that we already use JSON Web Token (JWT) for user claims, we chose to look at the IETF proposed JSON Object Signing and Encryption (JOSE) standard for our E2E encryption needs. We settled on using JSON Web Key (JWK) for key exchange and JSON Web Encryption (JWE) as our encryption format.

There are multiple libraries that implement the JOSE spec in different languages. We chose the following:

All three libraries have active communities, are well designed, have thoroughly documented APIs, and contain extensive test suites.

End-to-End Encryption in PixiePermalink

JWE supports a variety of key types and algorithms, however RSA-OAEP seems to be the most widely supported one across the many libraries. So we chose to use 4096 bit RSA keys with the RSA-OAEP encryption scheme across all our clients.

This is how a client interacts with Pixie after enabling end-to-end encryption.

The client generates an asymmetric keypair and sends the public key with any requests for data. Telemetry data is encrypted with the given public key on the cluster. It remains encrypted from the moment it leaves the cluster until it reaches the client.

The asymmetric keypairs are intentionally ephemeral and generated at client creation time and rotated across sessions. This lack of reuse of keys allows an additional layer of protection from any accidentally leaked private keys.

We encrypt all telemetry data. Other message fields currently remain unencrypted within the proxy and are used by the proxy to make routing decisions.

SummaryPermalink

Once we identified the various client libraries we wanted to use, implementing E2E encryption was straightforward. Check out the commits below for implementation details:

  • commit #1: Add encryption support to the Golang Server
  • commit #2: Add key creation & decryption support to the JavaScript UI
  • commit #3: Add key creation & decryption support to the Golang API
  • commit #4: Add key creation & decryption support to the Python API

We hope that the JOSE proposal becomes an IETF standard and this set of libraries and commits acts as a reference for anyone looking to implement E2E encryption in their own project!

Questions? Find us on Slack or Twitter at @pixie_run.


Related posts

Terms of Service|Privacy Policy

We are a Cloud Native Computing Foundation sandbox project.

CNCF logo

Pixie was originally created and contributed by New Relic, Inc.

Copyright © 2018 - The Pixie Authors. All Rights Reserved. | Content distributed under CC BY 4.0.
The Linux Foundation has registered trademarks and uses trademarks. For a list of trademarks of The Linux Foundation, please see our Trademark Usage Page.
Pixie was originally created and contributed by New Relic, Inc.

This site uses cookies to provide you with a better user experience. By using Pixie, you consent to our use of cookies.