Keywhiz is a system for managing and distributing secrets. It can fit well with a service oriented architecture (SOA). Here is an overview in presentation format
Every organization has services or systems that require secrets. Secrets like:
Common practices include putting secrets in config files next to code or copying files to servers out-of-band. The former is likely to be leaked and the latter difficult to track.
Keywhiz makes managing secrets easier and more secure. Keywhiz servers in a cluster centrally store secrets encrypted in a database. Clients use mutually authenticated TLS (mTLS) to retrieve secrets they have access to. Authenticated users administer Keywhiz via CLI. To enable workflows, Keywhiz has automation APIs over mTLS.
Keywhiz is reliable and used in production, however occasional changes may break API backward compatibility.
Keywhiz Server provides JSON APIs for accessing and managing secrets. It is written in Java and based on Dropwizard.
Keysync is a Keywhiz client that retrieves secrets from a Keywhiz Server using mTLS with a client certificate and stores them in tmpfs.
Presenting secrets as files makes Keywhiz compatible with nearly all software. Outside of Keywhiz administration, consumers of secrets only have to know how to read a file.
Keysync stores all secrets in memory only (via tmpfs) and never persists them to disk. If the server loses power, all secrets will be safely removed from that server.
Keysync's design mitigates a Keywhiz Server outage, since all necessary secrets are stored locally, and can be accessed even if Keysync process is restarted. The full sync of secrets is only necessary in the event of a server reboot.
Keywhiz CLI is a Java program for Keywhiz administration. Clients, secrets, and groups can be queried, added, removed, or associated with each other. Users can authenticate and use the CLI.
Keywhiz makes heavy use of mTLS and X509 certificates. It can even help distribute and rotate them for other services! There is the assumption of a PKI system though. If one does not exist or a PKI is wanted for development consider certstrap for a simple, initial PKI.
The data model is composed of clients, secrets, groups, and users.
Clients are identified by a client certificate. A database entry can enable/disable access or escalate automation privileges.
Secrets have a globally unique name and can contain any binary
content. For example, helloworldsrv-database.yml. Secrets are
immutable, but multiple ordered versions can exist at once. Arbitrary key-value
metadata can be added for extensibility.
Groups are the glue binding clients and secrets. Clients are assigned membership into groups (e.g. for the service-on-host, service, etc.). Secrets are also assigned to groups. If a client and secret share any group, the client will have access to the secret. Groups are not hierarchical and cannot be in other groups.
Users are authenticated Keywhiz administrators. Authentication is customizable with LDAP and bcrypt-hashed passwords currently supported. After initial authentication subsequent requests require an encrypted cookie from Keywhiz Server.
Source code for the Server and CLI is available in square/keywhiz. Keysync source is in square/keysync. To checkout the Keywhiz source code:
$ git clone https://github.com/square/keywhiz.git && cd keywhiz
Keywhiz stores data in MySQL and requires version 5.7 or higher.
From the base of the keywhiz repository, build the server:
$ mvn package -am -pl server
Run any migrations:
$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar preview-migrate server/src/main/resources/keywhiz-development.yaml $ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar migrate server/src/main/resources/keywhiz-development.yaml
Add an administrative user:
$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar add-user server/src/main/resources/keywhiz-development.yaml
Start the server:
$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar server server/src/main/resources/keywhiz-development.yaml
From the base of the keywhiz repository:
$ mvn package -am -pl cli
Run the CLI and get a usage statement:
$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar
In development, you can use --devTrustStore, e.g.:
$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar --devTrustStore --user keywhizAdmin list groups
You may want to alias this command for convenience:
$ alias keywhiz.cli="/path/to/keywhiz-cli-*-SNAPSHOT-shaded.jar"
Refer to the Keysync README.
More examples are available in the Wiki, but here are a few.
In development, you can use --devTrustStore and these examples
assume that you want to use the default keywhizAdmin user.
In production, you will want to create your own users and make sure that your certificates are properly signed and trusted.
$ keywhiz.cli --devTrustStore --user keywhizAdmin login $ keywhiz.cli add secret --name mySecretName < mySecretFile
The automation API requires a client certificate and
automationAllowed=true in the clients DB table. For
development purpose, you can use the pre-generated client.p12
keystore:
$ cat request.json
{
"name":"example.keytab",
"description":"example kerberos keytab",
"content":"a2V5dGFiIGNvbnRlbnQ=",
"metadata":{"owner":"root","group":"root","mode":"0400"}
}
$ curl --cert ./server/src/test/resources/clients/client.p12:ponies -H "Content-Type:application/json" -d @request.json https://localhost:4444/automation/secrets/
$ keywhiz.cli --devTrustStore --user keywhizAdmin assign secret --name example.keytab --group kerberos
$ curl --cert ./server/src/test/resources/clients/client.p12:ponies -X PUT "https://localhost:4444/automation/secrets/$SECRET_ID/groups/$GROUP_ID"
Source code for Keywhiz and this website is available on GitHub.
If you would like to contribute code to Keywhiz you can do so through GitHub by forking the repository and sending a pull request.
When submitting code, please make every effort to follow existing conventions
and style in order to keep the code as readable as possible. Please also make
sure your code compiles and passes tests by running mvn clean verify.
Before your code can be accepted into the project you must also sign the Individual Contributor License Agreement. We use cla-assistant.io and you will be prompted to sign once a pull request is opened.
Copyright 2015 Square, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
