Keywhiz is a system for managing and distributing secrets. It can fit well with a service oriented architecture (SOA). Here is an overview in presentation format
Every organization has services or systems that require secrets. Secrets like:
Common practices include putting secrets in config files next to code or copying files to servers out-of-band. The former is likely to be leaked and the latter difficult to track.
Keywhiz makes managing secrets easier and more secure. Keywhiz servers in a cluster centrally store secrets encrypted in a database. Clients use mutually authenticated TLS (mTLS) to retrieve secrets they have access to. Authenticated users administer Keywhiz via CLI. To enable workflows, Keywhiz has automation APIs over mTLS.
Keywhiz is reliable and used in production, however some upcoming changes may break API backward compatibility. See our roadmap.
Keywhiz Server provides JSON APIs for accessing and managing secrets. It is written in Java and based on Dropwizard.
KeywhizFs is a FUSE-based file system, providing secrets as if they are files in a directory. Transparently, secrets are retrieved from a Keywhiz Server using mTLS with a client certificate.
Presenting secrets as files makes Keywhiz compatible with nearly all software. Outside of Keywhiz administration, consumers of secrets only have to know how to read a file.
KeywhizFs stores all secrets in memory only and never persisted to disk. If KeywhizFs is unmounted or the server loses power, all secrets will be safely removed from that server.
To mitigate a Keywhiz Server outage, KeywhizFs maintains a local cache of previously accessed secrets. Unless the server is rebooted or KeywhizFs unmounted, applications can happily continue accessing secrets previously accessed.
Keywhiz CLI is a Java program for Keywhiz administration. Clients, secrets, and groups can be queried, added, removed, or associated with each other. Users can authenticate and use the CLI.
Keywhiz makes heavy use of mTLS and X509 certificates. It can even help distribute and rotate them for other services! There is the assumption of a PKI system though. If one does not exist or a PKI is wanted for development consider certstrap for a simple, initial PKI.
The data model is composed of clients, secrets, groups, and users.
Clients are identified by a client certificate. A database entry can enable/disable access or escalate automation privileges.
Secrets have a globally unique name and can contain any binary
content. For example, helloworldsrv-database.yml. Secrets are
immutable, but multiple ordered versions can exist at once. Arbitrary key-value
metadata can be added for extensibility.
Groups are the glue binding clients and secrets. Clients are assigned membership into groups (e.g. for the service-on-host, service, etc.). Secrets are also assigned to groups. If a client and secret share any group, the client will have access to the secret. Groups are not hierarchical and cannot be in other groups.
Users are authenticated Keywhiz administrators. Authentication is customizable with LDAP and bcrypt-hashed passwords currently supported. After initial authentication subsequent requests require an encrypted cookie from Keywhiz Server.
Source code for the Server and CLI is available in square/keywhiz. KeywhizFs source is in square/keywhiz-fs. To checkout the Keywhiz source code:
$ git clone https://github.com/square/keywhiz.git && cd keywhiz
Keywhiz can store data in MySQL or H2. H2 is the simplest database
for development purpose and all the data is stored in
/tmp/h2_data/keywhizdb_development. For production
systems you should use MySQL.
From the base of the keywhiz repository, build the server:
$ mvn package -am -pl server -P h2
Run any migrations:
$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar preview-migrate server/src/main/resources/keywhiz-development.yaml $ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar migrate server/src/main/resources/keywhiz-development.yaml
Add an administrative user:
$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar add-user server/src/main/resources/keywhiz-development.yaml
Start the server:
$ java -jar server/target/keywhiz-server-*-SNAPSHOT-shaded.jar server server/src/main/resources/keywhiz-development.yaml
From the base of the keywhiz repository:
$ mvn package -am -pl cli
Run the CLI and get a usage statement:
$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar
In development, you can use --devTrustStore, e.g.:
$ ./cli/target/keywhiz-cli-*-SNAPSHOT-shaded.jar --devTrustStore --user keywhizAdmin list groups
You may want to alias this command for convenience:
$ alias keywhiz.cli="/path/to/keywhiz-cli-*-SNAPSHOT-shaded.jar"
Refer to the KeywhizFs README.
More examples are available in the Wiki, but here are a few.
In development, you can use --devTrustStore and these examples
assume that you want to use the default keywhizAdmin user.
In production, you will want to create your own users and make sure that your certificates are properly signed and trusted.
$ keywhiz.cli --devTrustStore --user keywhizAdmin login $ keywhiz.cli add secret --name mySecretName < mySecretFile
The automation API requires a client certificate and
automationAllowed=true in the clients DB table. For
development purpose, you can use the pre-generated client.p12
keystore:
$ cat request.json
{
"name":"example.keytab",
"description":"example kerberos keytab",
"content":"a2V5dGFiIGNvbnRlbnQ=",
"metadata":{"owner":"root","group":"root","mode":"0400"}
}
$ curl --cert ./server/src/test/resources/clients/client.p12:ponies -H "Content-Type:application/json" -d @request.json https://localhost:4444/automation/secrets/
$ keywhiz.cli --devTrustStore --user keywhizAdmin assign secret --name example.keytab --group kerberos
$ curl --cert ./server/src/test/resources/clients/client.p12:ponies -X PUT "https://localhost:4444/automation/secrets/$SECRET_ID/groups/$GROUP_ID"
Source code for Keywhiz and this website is available on GitHub.
If you would like to contribute code to Keywhiz you can do so through GitHub by forking the repository and sending a pull request.
When submitting code, please make every effort to follow existing conventions
and style in order to keep the code as readable as possible. Please also make
sure your code compiles and passes tests by running mvn clean verify.
Before your code can be accepted into the project you must also sign the Individual Contributor License Agreement (CLA).
Copyright 2015 Square, Inc. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at https://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
