Apache ZooKeeper

Overview

Apache ZooKeeper GitHub Actions CI Travis CI Maven Central License

alt text

For the latest information about Apache ZooKeeper, please visit our website at:

https://zookeeper.apache.org

and our wiki, at:

https://cwiki.apache.org/confluence/display/ZOOKEEPER

Packaging/release artifacts

Either downloaded from https://zookeeper.apache.org/releases.html or found in zookeeper-assembly/target directory after building the project with maven.

apache-zookeeper-[version].tar.gz

    Contains all the source files which can be built by running:
    mvn clean install

    To generate an aggregated apidocs for zookeeper-server and zookeeper-jute:
    mvn javadoc:aggregate
    (generated files will be at target/site/apidocs)

apache-zookeeper-[version]-bin.tar.gz

    Contains all the jar files required to run ZooKeeper
    Full documentation can also be found in the docs folder

As of version 3.5.5, the parent, zookeeper and zookeeper-jute artifacts are deployed to the central repository after the release is voted on and approved by the Apache ZooKeeper PMC:

https://repo1.maven.org/maven2/org/apache/zookeeper/zookeeper

Java 8

If you are going to compile with Java 1.8, you should use a recent release at u211 or above.

Contributing

We always welcome new contributors to the project! See How to Contribute for details on how to submit patch through pull request and our contribution workflow.

Issues
  • ZOOKEEPER-3204: Reconfig tests are constantly failing on 3.5 after applying Java 11 fix

    ZOOKEEPER-3204: Reconfig tests are constantly failing on 3.5 after applying Java 11 fix

    Looks like that Netty 3 also having problems with correctly closing sockets and we keep getting "Bind failed" exceptions on the 3.5 branch.

    In this patch I'm trying to cherry-pick the upgrade-to-Netty4 commit by @ivmaykov from master branch and it looks like it solves the problem.

    I know this change is quite radical at this time of 3.5 release, but here're my 2 cents:

    • Netty 3 is rather old, would be a lot nicer to release 3.5 with a shiny new Netty lib support which will become stable in this release anyway (Netty is optional in 3.4)
    • After digging the net, I have absolutely no idea how to fix the Java11 binding issue with Netty 3. I suspect that the lib is as old as nobody bothers with the problem.

    Please take your time for a careful review.

    opened by anmolnar 50
  • ZOOKEEPER-3173: Quorum TLS - support PEM trust/key stores

    ZOOKEEPER-3173: Quorum TLS - support PEM trust/key stores

    Add support for loading key and trust stores from PEM files.

    Unfortunately, this PR includes 2 JIRAs, because it was quite difficult to untangle the two features as they were developed at the same time originally:

    • ZOOKEEPER-3173: Quorum TLS - support PEM trust/key stores
    • ZOOKEEPER-3175: Quorum TLS - test improvements

    Added support for PEM formatted key stores and trust stores

    • key store and trust store files can now be in PEM format as well as JKS.
    • Added config properties to tell ZK what type of trust/key store to load:
    • zookeeper.ssl.keyStore.type and zookeeper.ssl.trustStore.type for ClientX509Util
    • zookeeper.ssl.quorum.keyStore.type and zookeeper.ssl.quorum.trustStore.type for QuorumX509Util
    • store type properties could have the values "JKS", "PEM", or not set
    • leaving the type properties unset will cause auto-detection of the store type based on the file extension (".jks" or ".pem")

    Added test utilities for easily creating X509 certs and using them in unit tests

    • added new class X509TestContext and its friend, X509TestHelpers
    • rewrote some existing unit tests to use these classes, and added new tests that use them
    • some existing tests (i.e. QuorumSSLTest) should probably be ported to use this as well, haven't got around to it yet
    opened by ivmaykov 46
  • [ZOOKEEPER-1416] Persistent Recursive Watch

    [ZOOKEEPER-1416] Persistent Recursive Watch

    Here is a completed implementation for a persistent, recursive watch addition for ZK. These watches are set via a new method, addPersistentWatch() and are removed via the existing watcher removal methods. Persistent, recursive watches have these characteristics:

    • Once set, they do not auto-remove when triggered
    • They trigger for all event types (child, data, etc.) on the node they are registered for and any child znode recursively.
    • They are efficiently implemented by using the existing watch internals. A new class PathIterator walks up the path parent-by-parent when checking if a watcher applies.

    Persistent watcher specific tests are in PersistentWatcherTest.java. I'd appreciated feedback on other additional tests that should be added.

    opened by Randgalt 42
  • ZOOKEEPER-2122: add openssl to zookeeper c client to support ssl

    ZOOKEEPER-2122: add openssl to zookeeper c client to support ssl

    I have some problem with cmake, I was intend to find openssl as dependency using pkg_search_module, but it didn't work. so I have to set the include and lib directory in /opt/local, can anyone help me on this ? thanks.

    I made a wrong pull request to branch-3.5 #625, this one is a correction.

    opened by roodkcab 41
  • ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol

    ZOOKEEPER-236: SSL Support for Atomic Broadcast protocol

    This is a work in progress, I wanted to get some feedback from the community while I worked on this. Please do not merge yet. Tests, documentation, and some cleanup still coming.

    This is a first pass at ssl support for the zookeeper quorum. It supports encrypting both leader election and normal operation.

    Rolling upgrades are supported via port unification (portUnification=true). This should only be used while performing a rolling upgrade.

    Some open questions:

    • Anyone have any ideas for better names for the configuration options (sslQuorum and portUnification currently).
    • I am using the same configuration that points to the truststore/keystore used for server <-> client ssl. Do they need to be separate?
    • Is port unification the correct approach for rolling upgrades? Is the impact from the use of BufferedSockets during the upgrade acceptable? See: http://stackoverflow.com/questions/25637039/detecting-ssl-connection-and-converting-socket-to-sslsocket http://stackoverflow.com/questions/6559859/is-it-possible-to-change-plain-socket-to-sslsocket
    • server <-> client ssl is implemented with netty. I did not feel that rewriting our server <-> server logic with netty was necessary given how easy ssl was to implement with standard java SSLSockets. Any arguments to the contrary?

    Thanks, Abe

    opened by afine 39
  • ZOOKEEPER-3152: Port ZK netty stack to netty4

    ZOOKEEPER-3152: Port ZK netty stack to netty4

    Summary: Ported the client connection netty stack from netty3 to netty4. This includes both the server side (NettyServerCnxn and friends) and the client side (ClientCnxnSocketNetty).

    Test Plan: Modified FourLetterWordsTest and NettyServerCnxnTest, plus manual testing on a regional ensemble.

    FB Reviewers: nixon

    opened by ivmaykov 38
  • ZOOKEEPER-2184: Zookeeper Client should re-resolve hosts when connection attempts fail

    ZOOKEEPER-2184: Zookeeper Client should re-resolve hosts when connection attempts fail

    This is a version of the patch for ZK-2184 for the 3.4 branch, compatible with Java 6.

    opened by fpj 34
  • ZOOKEEPER-3731: Disallow HTTP TRACE method on PrometheusMetrics Server

    ZOOKEEPER-3731: Disallow HTTP TRACE method on PrometheusMetrics Server

    This is a copy of https://github.com/apache/zookeeper/pull/1349/files (ZOOKEEPER-3772) but for the PrometheusMetrics Server.

    Added the test requested in https://github.com/apache/zookeeper/pull/1539

    opened by doxsch 0
  • ZOOKEEPER-4021: Solve Poll timeout failure  caused by POLLNVAL

    ZOOKEEPER-4021: Solve Poll timeout failure caused by POLLNVAL

    Zookeeper c client request server by dns , when the zookeeper server and dns server down at the same time , c client poll will set revent to POLLNVAL . At this time The parameter timeout of poll is not valid. it cause very frequent requests and high cpu use

    opened by qwedsazzcc 0
  • ZOOKEEPER-4276: Support setupQuorumPeerConfig only with secureClientPort

    ZOOKEEPER-4276: Support setupQuorumPeerConfig only with secureClientPort

    clientPort in zoo.cfg is forcefully complemented from client address by QuorumPeerConfig#setupClientPort even though secureClientPort is set and matches with client address' port. Because of this behavior, in case rolling update with replacing clientPort to secureClientPort in the same port number following Upgrading existing non-TLS cluster with no downtime conflicts and gets errors below.

    2021-03-29 23:21:58,638 - INFO  [main:[email protected]] - binding to port /0.0.0.0:2181
    2021-03-29 23:21:58,748 - INFO  [main:[email protected]] - bound to port 2181
    2021-03-29 23:21:58,749 - INFO  [main:[email protected]] - binding to port 0.0.0.0/0.0.0.0:2181
    2021-03-29 23:21:58,753 - ERROR [main:[email protected]] - Unexpected exception, exiting abnormally
    java.net.BindException: Address already in use
    

    This PR makes clientPort complement by QuorumPeerConfig#setupClientPort only when both clientPort and secureClientPort are empty, and allow serving zookeeper server only with secure client port.

    opened by 2k0ri 1
  • ZOOKEEPER-3754: Add OSGi support

    ZOOKEEPER-3754: Add OSGi support

    Added the maven bundle plugin to the pom, and an OSGi integration test (pax exam) to make sure it really works.

    Before merging this I just want to verify a couple things with the devs:

    1. The dependencies marked as optional vs required are correct.
    2. The zookeeper-jute dependency module seems to contain the same package names as the zookeeper-server module, which is bad practice in general and troublesome in OSGi in particular. I'm not sure what jute is, but to avoid issues I thought maybe it's better to embed it within the server bundle rather than import it as an external dependency (via the Embed-Dependency configuration)... what do you think?
    opened by amichair 0
  • Zookeeper-4271: Flaky test - ReadOnlyModeTest.testConnectionEvents

    Zookeeper-4271: Flaky test - ReadOnlyModeTest.testConnectionEvents

    Some refactoring and cleanup, followed by the fix - client wasn't properly shut down between tests so occasionally they contaminated each other with unwanted events.

    opened by amichair 0
  • ZOOKEEPER-4267: Fix check-style issues

    ZOOKEEPER-4267: Fix check-style issues

    Formatted CertificatesToPlayWith to fix check-style issue :(whitespace) OperatorWrap: '+' should be on a new line

    opened by arshadmohammad 1
Owner
The Apache Software Foundation
The Apache Software Foundation
Apache Geode

Contents Overview How to Get Apache Geode Main Concepts and Components Location of Directions for Building from Source Geode in 5 minutes Application

The Apache Software Foundation 1.9k Mar 12, 2021
Mirror of Apache Storm

Master Branch: Storm is a distributed realtime computation system. Similar to how Hadoop provides a set of general primitives for doing batch processi

The Apache Software Foundation 6.2k Mar 12, 2021
Distributed Stream and Batch Processing

What is Jet Jet is an open-source, in-memory, distributed batch and stream processing engine. You can use it to process large volumes of real-time eve

hazelcast 850 Mar 11, 2021
Netflix, Inc. 21.2k Mar 13, 2021
Resilience4j is a fault tolerance library designed for Java8 and functional programming

Fault tolerance library designed for functional programming Table of Contents 1. Introduction 2. Documentation 3. Overview 4. Resilience patterns 5. S

Resilience4j 6.6k Mar 13, 2021
APM, (Application Performance Management) tool for large-scale distributed systems.

Visit our official web site for more information and Latest updates on Pinpoint. Latest Release (2020/01/21) We're happy to announce the release of Pi

null 11.2k Mar 13, 2021
Ribbon is a Inter Process Communication (remote procedure calls) library with built in software load balancers. The primary usage model involves REST calls with various serialization scheme support.

Ribbon Ribbon is a client side IPC library that is battle-tested in cloud. It provides the following features Load balancing Fault tolerance Multiple

Netflix, Inc. 3.9k Mar 11, 2021
Fault tolerance and resilience patterns for the JVM

Failsafe Failsafe is a lightweight, zero-dependency library for handling failures in Java 8+, with a concise API for handling everyday use cases and t

Jonathan Halterman 3.4k Mar 11, 2021
Reactive Microservices for the JVM

Lagom - The Reactive Microservices Framework Lagom is a Swedish word meaning just right, sufficient. Microservices are about creating services that ar

Lagom Framework 2.5k Mar 12, 2021
Open Source In-Memory Data Grid

Hazelcast Hazelcast is an open-source distributed in-memory data store and computation platform. It provides a wide variety of distributed data struct

hazelcast 4.2k Mar 12, 2021
A reactive dataflow engine, a data stream processing framework using Vert.x

?? NeonBee Core NeonBee is an open source reactive dataflow engine, a data stream processing framework using Vert.x. Description NeonBee abstracts mos

SAP 14 Apr 1, 2021
Build highly concurrent, distributed, and resilient message-driven applications on the JVM

Akka We believe that writing correct concurrent & distributed, resilient and elastic applications is too hard. Most of the time it's because we are us

Akka Project 11.4k Mar 13, 2021
BitTorrent library and client with DHT, magnet links, encryption and more

Bt A full-featured BitTorrent implementation in Java 8 peer exchange | magnet links | DHT | encryption | LSD | private trackers | extended protocol |

Andrei Tomashpolskiy 2k Mar 13, 2021