tl;dr We have received our SOC 2 Type I report, certifying that a third party auditor has confirmed that we back our security and privacy stance, complete with vulnerability scans and penetration tests. We have always taken user data security seriously, from end-to-end encryption to never training our generative autocomplete model on user code.

What is SOC 2 and Why is it Important?

SOC 2 is a third party audit that verifies that a company meets their stated standards across security, availability, confidentiality, processing integrity, and privacy, especially surrounding customer data. It is important as it is the only unbiased way to prove policies are being followed.

More broadly, SOC 2 is the type of System and Organization Controls (SOC) reports that deals with information security, acting as a third party assessment covering five categories that form what is called the Trust Service Criteria: security, availability, confidentiality, processing integrity, and privacy. These criteria comprehensively establish protection against unauthorized access, availability of the service in terms of both performance and disaster/incident recovery, availability of information on only a need-to-know basis internally, monitoring and assurance, and protection of personal information.

A third-party audit is the only way to prove that companies are following their stated policies. The auditor performs a full assessment of the product and the vendor, including vulnerability scans and penetration tests of the system. While companies can self-report the policies they have for these criteria, a SOC 2 attestation is the only proof that a third party has reviewed and affirmed that these policies are actually followed. Using a vendor without SOC 2 attestation is equivalent to taking the vendor on their word, and any gaps in their policies are often only discoverable once a major breach or issue happens, which is often too late.

Codeium is SOC 2 Compliant

SOC 2 mainly applies to our Codeium-hosted deployment, and we are proud to announce that we have successfully passed the third-party audit performed by Prescient Assurance. The Codeium-hosted deployment is what underlies both our free individual plan and our upcoming Teams tier, a SaaS solution for enterprises. Since we are processing user data, we knew that we wanted to get a third party auditor to verify our security and privacy posture.

However, if you want more peace of mind, our second deployment option for enterprises is done fully within an enterprise’s tenant, which means all processing and data never leaves externally, even to Codeium. This is great for companies with strict self-hosting requirements. So while this deployment method does not necessarily require SOC 2 compliance (no data or processing happens on Codeium’s end for us to have a SOC 2 attestation over, and an enterprise can verify that no data leaves their firewall), this SOC 2 compliance should further increase confidence in this system. The nature of this deployment is why we have gotten approval from legal and infosec teams across defense, finance, healthcare, and more on this plan, even before we received SOC 2 compliance for our Codeium-hosted plans!

At Codeium, we have always had a very strong stance on security & privacy. We never train our proprietary generative autocomplete model on user data, encrypt all data in transit and rest, provide opt-outs from code snippet telemetry, and more. Now, Codeium is SOC 2 compliant, proving that we back our talk.

Compliance in Generative AI Tools for Software Development

Codeium, just like with legal compliance, is leading the space on security compliance. The only other enterprise AI assistant that has similar levels of security compliance is Tabnine, but we have discussed that it did not have the level of quality or important features to be a useful tool.

On the other hand, tools such as GitHub Copilot, Amazon CodeWhisperer, and Sourcegraph Cody all have varying levels of security promises with no third-party compliance or audit performed.

GitHub Copilot

While GitHub does claim zero day retention and other security features, GitHub Copilot has NOT been audited by a third party to verify and test these claims. So while GitHub’s core product is SOC 2 compliant (among others), GitHub Copilot explicitly is not, and there haven’t been any external penetration tests conducted. This is very different from tools such as Codeium Teams or Tabnine, which are SOC 2 compliant. GitHub Copilot will NOT have this compliance for a while according to their own trust center FAQ:

Audits and Certifications: GitHub Copilot is not currently included in GitHub’s existing audits and certifications, including SOC 2, ISO 27001, and FedRAMP Tailored. Compliance at GitHub begins with good security, so our first focus is fully onboarding Copilot to GitHub security programs and tooling. GitHub is engaging with a third-party audit firm to perform a gap assessment of Copilot as part of readiness activities for SOC 2 Type 1 (security criteria) and ISO 27001, with a goal of having the full audits completed by May 2024.

External Penetration Test: GitHub has not yet performed an external penetration test of Copilot. GitHub plans to include Copilot for Business penetration testing in the next cycle, which is in the 2nd half of 2023, with the report being issued in early 2024.

Amazon CodeWhisperer

There is limited documentation around what data Amazon CodeWhisperer collects and retains, but what is clear from AWS’s Compliance Portal that Amazon CodeWhisperer does not currently have SOC 2 compliance. There is no word on when, or even if, Amazon will perform a third party audit and receive attestation.

Sourcegraph Cody

Sourcegraph Cody does use OpenAI’s and Anthropic’s APIs for their model layers. While Anthropic is SOC 2 compliant, Sourcegraph Cody is not SOC 2 compliant. Not only are they not SOC 2 compliant, they actually quite explicitly say that they would store your code snippets and data if you use sourcegraph.com or Sourcegraph Cloud, and they have usage data and feedback to directly improve the product. So, while Sourcegraph Cody uses a third party LLM that is SOC2 compliant and guarantees zero day retention, Sourcegraph themselves do not abide by these rules and audits, and there is no word on when, or if, they will perform a third party audit and receive attestation.

Looking Forwards on Compliance

We are already working on getting even stronger compliance audits. We have received SOC 2 Type I attestation, which is a verification of our policies, and we have already started working on our SOC 2 Type II attestation, which will further verify our policies over an extended period of time. We will remain the most secure and compliant generative AI platform for software developers, whether managed by Codeium or self-hosted within an enterprise’s environment.

Use a code assistant that you can trust: