GoReplay is an open-source tool for capturing and replaying live HTTP traffic into a test environment in order to continuously test your system with real data. It can be used to increase confidence in code deployments, configuration changes and infrastructure changes.

Overview

GitHub release codebeat Go Report Card Join the chat at https://gitter.im/buger/gor Reviewed by Hound

Go Replay

https://goreplay.org/

GoReplay is an open-source network monitoring tool which can record your live traffic, and use it for shadowing, load testing, monitoring and detailed analysis.

About

As your application grows, the effort required to test it also grows exponentially. GoReplay offers you the simple idea of reusing your existing traffic for testing, which makes it incredibly powerful. Our state of art technique allows you to analyze and record your application traffic without affecting it. This eliminates the risks that come with putting a third party component in the critical path.

GoReplay increases your confidence in code deployments, configuration and infrastructure changes.

GoReplay offers unique approach for shadowing. Instead of being a proxy, GoReplay in background listen for traffic on your network interface, requiring no changes in your production infrastructure, rather then running GoReplay daemon on the same machine as your service.

Diagram

Check latest documentation.

Installation

Download latest binary from https://github.com/buger/goreplay/releases or compile by yourself.

Getting started

The most basic setup will be sudo ./gor --input-raw :8000 --output-stdout which acts like tcpdump. If you already have test environment you can start replaying: sudo ./gor --input-raw :8000 --output-http http://staging.env.

See the our documentation and Getting started page for more info.

Newsletter

Subscribe to our newsletter to stay informed about the latest features and changes to Gor project.

Want to Upgrade?

We have created a GoReplay PRO extension which provides additional features such as support for binary protocols like Thrift or ProtocolBuffers, saving and replaying from cloud storage, TCP sessions replication, etc. The PRO version also includes a commercial-friendly license, dedicated support, and it also allows you to support high-quality open source development.

Problems?

If you have a problem, please review the FAQ and Troubleshooting wiki pages. Searching the issues for your problem is also a good idea.

All bug-reports and suggestions should go through Github Issues or our Google Group (you can just send email to [email protected]). If you have a private question feel free to send email to [email protected].

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Companies using Gor

  • GOV.UK - UK Government Digital Service
  • theguardian.com - Most popular online newspaper in the UK
  • TomTom - Global leader in navigation, traffic and map products, GPS Sport Watches and fleet management solutions.
  • 3SCALE - API infrastructure to manage your APIs for internal or external users
  • Optionlab - Optimize customer experience and drive engagement across multiple channels
  • TubeMogul - Software for Brand Advertising
  • Videology - Video advertising platform
  • ForeksMobile - One of the leading financial application development company in Turkey
  • Granify - AI backed SaaS solution that enables online retailers to maximise their sales
  • And many more!

If you are using Gor, we are happy to add you to the list and share your story, just write to: [email protected]

Author

Leonid Bugaev, @buger, https://leonsbox.com

Issues
  • How to merge/split  traffic from multiple endpoints

    How to merge/split traffic from multiple endpoints

    We have multiple endpoints for HA/load balancing reasons. To reconstruct production traffic recorded on all those nodes we would need to merge all recorded files in to one.

    I noticed in recording file timestamps are not in exact order but seems to be played in order. Is that right ?

    Will concatenating and sorting all records by timestamp do the job ?

    Also in the future we would like to split merged file to run on N clients to emulate real clients.

    We plan to split by sessions/source IP. Ideas ?

    I believe we are not first who tries to do this. I know google groups would be more appropriate , just do not see any activity there.

    opened by nikolai-derzhak-distillery 34
  • Middleware for custom request rewrite logic

    Middleware for custom request rewrite logic

    Middleware

    Middleware is a program that accepts request and response payload at STDIN and emits modified requests at STDOUT. You can implement any custom logic like stripping private data, advanced rewriting, support for oAuth and etc.

                       Original request      +--------------+
    +-------------+----------STDIN---------->+              |
    |  Gor input  |                          |  Middleware  |
    +-------------+----------STDIN---------->+              |
                       Original response     +------+---+---+
                                                    |   ^
    +-------------+    Modified request             v   |
    | Gor output  +<---------STDOUT-----------------+   |
    +-----+-------+                                     |
          |                                             |
          |            Replayed response                |
          +------------------STDIN----------------->----+
    

    Middleware can be written in any language, see examples/middleware folder for examples. Middleware program should accept the fact that all communication with Gor is asynchronous, there is no guarantee that original request and response messages will come one after each other. Your app should take care of the state if logic depends on original or replayed response, see examples/middleware/token_modifier.go as example.

    Simple bash echo middleware (returns same request) will look like this:

    while read line; do
      echo $line
    end
    

    Middleware can be enabled using --middleware option, by specifying path to executable file:

    gor --input-raw :80 --middleware "/opt/middleware_executable" --output-http "http://staging.server"
    

    Communication protocol

    All messages should be hex encoded, new line character specifieds the end of the message, eg. new message per line.

    Decoded payload consist of 2 parts: header and HTTP payload, separated by new line character.

    Example request payload:

    1 932079936fa4306fc308d67588178d17d823647c 1439818823587396305
    GET /a HTTP/1.1
    Host: 127.0.0.1
    
    

    Example response payload:

    2 8e091765ae902fef8a2b7d9dd960e9d52222bd8c 2782013
    HTTP/1.1 200 OK
    Date: Mon, 17 Aug 2015 13:40:23 GMT
    Content-Length: 0
    Content-Type: text/plain; charset=utf-8
    
    

    Header contains request meta information separated by spaces. First value is payload type, possible values: 1 - request, 2 - original response, 3 - replayed response. Next goes request id: unique among all requests (sha1 of time and Ack), but remain same for original and replayed response, so you can create associations between request and responses. Third argument varies depending on payload type: for request - start time, for responses - round-trip time.

    HTTP payload is unmodified HTTP requests/responses intercepted from network. You can read more about request format here, here and here. You can operate with payload as you want, add headers, change path, and etc. Basically you just editing a string, just ensure that it is RCF compliant.

    At the end modified (or untouched) request should be emitted back to STDOUT, keeping original header, and hex-encoded. If you want to filter request, just not send it. Emitting responses back is optional, and does not affect anything at the moment.

    Advanced example

    Imagine that you have auth system that randomly generate access tokens, which used later for accessing secure content. Since there is no pre-defined token value, naive approach without middleware (or if middleware use only request payloads) will fail, because replayed server have own tokens, not synced with origin. To fix this, our middleware should take in account responses of replayed and origin server, store originalToken -> replayedToken aliases and rewrite all requests using this token to use replayed alias. See examples/middleware/token_modifier.go and middleware_test.go#TestTokenMiddleware as example of described scheme.

    Latest precompiled binaries: https://www.dropbox.com/s/27chmbsqxrolvz4/gor_0.9.9_middleware_x64.tar.gz?dl=1 https://www.dropbox.com/s/vmeg5sexcleoo2e/gor_0.9.9_middleware_x86.tar.gz?dl=1

    opened by buger 28
  • Using same input and output port causes request duplication

    Using same input and output port causes request duplication

    Hello! Amazing work you did here. However, we found a small issue at our company.

    Using gor with this commandline: gor --output-http-timeout '1800s' --input-raw 'intf-ip-addr:8080' --output-http 'http://output-http-addr:8080|100%' --http-allow-url '/some/url' causes requests to be replayed multiple times on target machine (as checked on tomcat access-logs and tcpdump). Replaying request to another port (eg. 8081) fixes the issue. We confirmed the issue on gor versions 0.12.1 and 0.13.0, however, issue is not present in version 0.10.1. Machine runs binary release of gor on CentOS 6.6 2.6.32-504.12.2.el6.x86_64.

    opened by walczakp 26
  • --output-http-track-response  not returning response for replayed requests.

    --output-http-track-response not returning response for replayed requests.

    Hi here,

    I am trying to capture responses of the replayed requests , but not able to get back responses all time . On replaying a request, the program code is getting stuck on the net.conn.Read() call. https://github.com/buger/goreplay/blob/master/http_client.go#L199 .

    I see the response in tcpdump output, but not able to obtain the same from within the code using net.conn.Read()

    opened by jacoblukose 22
  • Errors: dial tcp: cannot assign requested address

    Errors: dial tcp: cannot assign requested address

    Hi, I'm seeing many errors from gor:

    2014/09/04 16:20:20 Request error: Get http://....: dial tcp x.x.x.x:8080: cannot assign requested address
    

    (x.x.x.x is an IP address and the actual http url removed).

    I use gor as:

    sudo ./gor --input-raw :80 --output-tcp y.y.y.y:28020
    

    and on y.y.y.y:

    ./gor --input-tcp :28020 --output-http http://x.x.x.x:8080
    

    The error seem to stem from high load in some sense. When request rate is low they are gone and everything works as expected. It's possible that the accepting http server (x.x.x.x:8080) is slow on high load. Could that be the reason for the errors on gor? Anyway, it's more of a hunch. What do these errors actually mean? And how to mitigate them? (even assuming that the accepting server is indeed slow)

    I tried, out of the blue to increase ulimit and set GOMAXPROCS to various numbers but these are just shots in the dark and as expected, they didn't help.

    What's your advise?

    (btw, sorry if there's a mailing list I didn't find it, so I'm posting this as a bug here, it doesn't mean I think it's an actual bug with gor of course).

    opened by rantav 21
  • Is the request id unique globally?

    Is the request id unique globally?

    As the document, I expect it is. image

    But after I captured traffic and analyze the .gor file, I saw that request id is not seemly unique. The following picture is my traffic file and I searched one request Id but 46 matched.

    image

    And I found that tcp_message.go#UUID() function calculates the request id by src address and dest address, which seems not to be unique.

    Could you please answer this question? Thanks!

    opened by ZmfCn 20
  • Limit listening side

    Limit listening side

    Hi,

    I have a volume problem where we get up to 20k qps on the server I need to sniff traffic from but i only need to forward a very small subset of them onto our test servers, ideally around 5-10 qps. I have tried limiting on the replay side but each time I start up I currently see the following error: "Error while Accept() accept tcp [::]:28020: too many open files"

    I believe if there is a way to limit as well on the sending side then this would not be an issue.

    Thanks

    Mark

    opened by pixie79 20
  • Gor performance issues

    Gor performance issues

    Don't know which would be the best way to debug GOR for performance testing. We've been working with it for almost a month now, and I've seen an increase on the load avg on a canary 300+ req/s aprox. It reach to a point where not only gor as a listener but the replay begin to grow instance loads.

    I am talking about m1.large instances on AWS 8GB ram, 2 cores Instance Store, yesterday someone mentioned that maybe it could be related to encoding characters on our requests (because we have japanese characters) but I think it doesn't make any sense.

    What you think @buger? any test that I can run on my systems to detect the bottleneck? and we are running 0.7.5.

    opened by rhoml 20
  • elasticsearch can't parse URI

    elasticsearch can't parse URI

    hi there,

    Based on your doc via https://github.com/buger/goreplay/wiki/Exporting-to-ElasticSearch

    The es7 and gor 1.2 have been installed in my centos7 OS and, I ran the command as below: (Is there any version limitation of elasticsearch?)

    /usr/local/bin/gor --input-raw :8080 --output-http http://localhost:8081 --output-http-elasticsearch http://localhost:9200/test123

    The test123 index has not been created. Also, the pattern of es URI must be scheme://host/index_name Otherwise, the exception will be thrown out: 2020/11/13 15:49:38 Can't initialize ElasticSearch plugin.Wrong ElasticSearch URL format. Expected to be: scheme://host/index_name

    It is appreciated that you could guide me on how to resolve this issue.

    And, another issue is about kafka with version 1.2, which https://github.com/buger/goreplay/issues/848 has been submitted. I am concerning whether this bug related with kafka will be fixed in next version (v1.3?) and when will the next version be released?

    Sorry for your inconvenience. BR Kimi

    needs info 
    opened by quzhixue-Kimi 18
  • Can't rewrite Host header

    Can't rewrite Host header

    This is on 0.7.5

    ~#  ./gor --input-raw :80  --output-http "http://my-staging-host"  --output-http-header "Host: my-staging-host"
    

    But.. sniffing the traffic

    ~# tcpdump -c 20 -s 1023 -A host my-staging-host and tcp port http
    ...
    Host: news-business.vlex.com
    User-Agent: Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
    ...
    Host: borme.vlex.es
    Connection: Keep-alive
    Accept: */*
    

    so it seems that Hostname is not being rewritten

    opened by angelf 18
  • Makefile fixes2

    Makefile fixes2

    @buger fixed to my commit about the makefile the vendor target needs to be PHONY

    • change sleep in the test to two seconds. i think 200ms is to low and the test does not pass
    opened by DimaGolomozy 1
  • --output-http-track-response not returning response for few transactions

    --output-http-track-response not returning response for few transactions

    Hello,

    I am trying to log requests and responses data to a file, but not able to get responses for few transactions.

    Below is the command: ./gor --input-raw :80 --input-raw-track-response --output-file %Y%m%d.log --output-file-append

    Thanks, Surya

    opened by suryapadarthi 0
  • Request filtering not working

    Request filtering not working

    [[email protected] ~]# ./gor
    2021/09/02 18:11:46 [PPID 47468 and PID 12540] Version:1.3.0
    2021/09/02 18:11:46 Required at least 1 input and 1 output
    [[email protected] ~]# uname -a
    Linux armpvm 3.10.0-1160.6.1.el7.x86_64 #1 SMP Tue Nov 17 13:59:11 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
    

    image

    Any --http-allow-xxxx option not working for me , just like in the screenshot I clearly filtered the GET method,but the POST method is also being dumped. Is it because I'm not using it in the right way?

    opened by almas1992 0
  • gor 1.2.0 --http-allow-url cannot be filtered correctly

    gor 1.2.0 --http-allow-url cannot be filtered correctly

    --http-allow-url cannot filter out non-allow URLs correctly, there will be a small number of non-allow paths to write files

    opened by wjyGithub 0
  • gor traffic copy write file error, partly out of order

    gor traffic copy write file error, partly out of order

    image As shown above, Access-Token contains the next request line information

    opened by wjyGithub 5
  • replay    Error when sending  EOF

    replay Error when sending EOF

    command: gor120 -input-file /opt/nginx01/23/*|4000% -output-http-workers=5000 -output-http=https://api-pts.xxx.com -http-pprof :8181 -output-http-timeout 30s -http-disallow-url /api/health -http-disallow-url /api/ext/xxx -verbose 1

    Current Version: v1.2.0

    Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 487.591µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 16.364µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 1.419066ms]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 1.297174ms]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 15.242µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 7.642µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 170.785µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 6.566µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 181.862µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 289.085µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 8.023µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 735.448µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 1.2466ms]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 13.042µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 11.35µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 1.095554ms]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 23.653µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 1.410718ms]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 11.825µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 23.468µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 7.081µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 16.843µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 82.386µs]: Error when sending EOF Aug 30 15:37:01 pts_10_20_20_62 gorc[21038]: [DEBUG][elapsed 5.006µs]: Error when sending EOF

    opened by arrow2012 1
  • Specify Kafka Offset

    Specify Kafka Offset

    When i use --input-kafka-host 'xxx' --input-kafka-topic 'xxx', i want to specify the kafka offset, how could i config it?

    opened by ailun123456789 0
  • package gor-1.3.2-1.x86_64 is intended for a different operating system

    package gor-1.3.2-1.x86_64 is intended for a different operating system

    Linux

    # cat /etc/redhat-release
    CentOS Linux release 7.5.1804 (Core)
    
    # uname -a
    Linux myhost 3.10.0-862.6.3.el7.x86_64 #1 SMP Tue Jun 26 16:32:21 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
    

    rpm errr

    # rpm -ivh gor-1.3.2-1.x86_64.rpm
    Preparing...                          ################################# [100%]
    	package gor-1.3.2-1.x86_64 is intended for a different operating system
    

    RPM package is not avaliable for centos 7.x

    opened by opsarno 1
  • Feature/config management

    Feature/config management

    fix conflict of https://github.com/buger/goreplay/pull/874

    feature : limiter bind to service stats bind to service kafka-input debug log of success message don't fatal when kafka-plugin construct failed

    opened by xingren23 4
Releases(v1.3.2)
  • v1.3.2(Aug 19, 2021)

    • Fix promiscuous mode - if enabled no need to filter on host as we want to capture traffic not related to this host #994
    • Fix --input-raw-realip-header https://github.com/buger/goreplay/pull/996
    • Fixed false positive network interface lookups https://github.com/buger/goreplay/pull/1000

    Thanks @DimaGolomozy!

    Source code(tar.gz)
    Source code(zip)
    gor-1.3.2_windows.zip(7.57 MB)
    gor-1.3.2.pkg(10.93 MB)
    gor_1.3.2_mac.tar.gz(10.93 MB)
    gor-1.3.2-1.x86_64.rpm(10.51 MB)
    gor_1.3.2_amd64.deb(10.55 MB)
    gor_1.3.2_x64.tar.gz(10.55 MB)
  • v1.3.1(Aug 12, 2021)

  • v1.3.0(Aug 6, 2021)

    Highlights

    Added official Windows support https://github.com/buger/goreplay/wiki/Running-on-Windows

    Traffic capture engine fully re-constructed to be more scalable, efficient and bulletproof. You should see way lower packet loss, capture quality, and CPU/Memory usage.

    Deprecated own HTTP client in favour of native Go HTTP library https://github.com/buger/goreplay/pull/833

    Increased stability, and numerous bug fixes.

    Features

    • Added way to capture traffic from multipe ports with single input --input-raw :80,8080,3000 https://github.com/buger/goreplay/pull/935
    • Added --output-tcp-skip-verify flag to skip TCP output TLS verification https://github.com/buger/goreplay/pull/814 @tomerf
    • Added --output-tcp-workers to configure number of TCP output connections https://github.com/buger/goreplay/pull/819
    • Added new debugging endpoint when --http-pprof enabled: /debug/vars
    • Added new %i file name variable to inject unique instance ID to file name. Can be useful if you have logs from multiple machines and you need unique names.
    • Added --input-file-dry-run option to previewing how long it will take and how many requests in given file input https://github.com/buger/goreplay/commit/8e76559b492402ef1d2aee99e0106d085db7f935
    • Added --input-raw-max-wait option (in seconds), which allows to skip long pauses in recorded files https://github.com/buger/goreplay/commit/19ad90aa4bb0320c8ba623d5509a0690e8c3ec25
    • Added --input-file-read-depth option to pre-read and buffer requests (and sort them). By default, is 100 https://github.com/buger/goreplay/commit/625ed54f1ebcc006e7d04a27544807fbaff08508
    • Added --input-raw-timestamp-type go to enable application level timestamps, when network timestamps unreliable https://github.com/buger/goreplay/commit/8edb74e57276c712444ccfb89ed40dd0ca5c39d6

    Thank you, @urbanishimwe, @slimus, @arijitAD for jumping with me to this adventure!

    Huge kudos to all GoReplay community, especially to our contributors @DimaGolomozy @tomerf @swills @davidFR @two @jl2005 @rmasclef @othererik @ankitdobhal @eko @code4wt @wangfeng22 @lins05 @betty1126 @YuriYin @StanleyP

    Source code(tar.gz)
    Source code(zip)
    gor-1.3.0_windows.zip(7.57 MB)
    gor-1.3.0.pkg(10.93 MB)
    gor_1.3.0_mac.tar.gz(10.93 MB)
    gor-1.3.0-1.x86_64.rpm(10.51 MB)
    gor_1.3.0_amd64.deb(10.55 MB)
    gor_1.3.0_x64.tar.gz(10.55 MB)
  • v1.3.0_RC11(Aug 3, 2021)

    • Fix loopback interface issue on windows https://github.com/buger/goreplay/commit/c9274ac92a6f021240d82682002240cfceaecd5e
    • Fix buffer index number for sticky tcp output connection https://github.com/buger/goreplay/pull/981 @DimaGolomozy
    • Fix output file size limits https://github.com/buger/goreplay/pull/974 @lins05
    • Fixed usage of --input-raw-bpf-filter https://github.com/buger/goreplay/commit/2b993eda49b2a9e9d54ae6cbd616dbb6f6f28388
    • Fix BPF filter when listen on all interfaces https://github.com/buger/goreplay/commit/a727ade00608b526f0388e9d60c21705136be75c
    • Optimize packet capture thread to reduce context switching https://github.com/buger/goreplay/commit/214edb45f63ac38dc63c9f21017ddfab557f9d06
    • Added new debugging endpoint when --http-pprof enabled: /debug/vars
    • Added --input-file-dry-run option to previewing how long it will take and how many requests in given file input https://github.com/buger/goreplay/commit/8e76559b492402ef1d2aee99e0106d085db7f935
    • Added --input-raw-max-wait option (in seconds), which allows to skip long pauses in recorded files https://github.com/buger/goreplay/commit/19ad90aa4bb0320c8ba623d5509a0690e8c3ec25
    • Added --input-file-read-depth option to pre-read and buffer requests (and sort them). By default, is 100 https://github.com/buger/goreplay/commit/625ed54f1ebcc006e7d04a27544807fbaff08508
    • Added --input-raw-timestamp-type go to enable application level timestamps, when network timestamps unreliable https://github.com/buger/goreplay/commit/8edb74e57276c712444ccfb89ed40dd0ca5c39d6
    Source code(tar.gz)
    Source code(zip)
    gor-1.3.0_RC11_windows.zip(7.57 MB)
    gor-1.3.0_RC11.pkg(10.93 MB)
    gor_1.3.0_RC11_mac.tar.gz(10.93 MB)
    gor-1.3.0_RC11-1.x86_64.rpm(10.51 MB)
    gor_1.3.0_RC11_amd64.deb(10.55 MB)
    gor_1.3.0_RC11_x64.tar.gz(10.55 MB)
  • v1.3.0_RC6(Jun 30, 2021)

    • Significantly improved CPU and Memory usage.
    • Added af_packet packet capture engine for Linux https://github.com/buger/goreplay/pull/950
    • Skip incomplete or malformed messages from output (can be enabled back by setting --input-raw-allow-incomplete) https://github.com/buger/goreplay/pull/953
    Source code(tar.gz)
    Source code(zip)
    gor-v1.3.0_RC6_windows.zip(7.55 MB)
    gor-v1.3.0_RC6.pkg(10.89 MB)
    gor_v1.3.0_RC6_mac.tar.gz(10.89 MB)
    gor-v1.3.0_RC6-1.x86_64.rpm(10.48 MB)
    gor_v1.3.0_RC6_amd64.deb(10.53 MB)
    gor_v1.3.0_RC6_x64.tar.gz(10.53 MB)
  • v1.3.0_RC5(Jun 10, 2021)

    • Added a way to capture multiple ports using one input like this: --input-raw :80,8080,3000 #935
    • Fixed usage of "localhost" hostname when trying to limit scope of packet capture (e.g. --input-raw localhost:80), now it properly binds to loopback interface #943
    • Fixed detection of windows network interfaces #943
    • Fixed Makefile job to generate windows binaries
    • Fixed big memory allocation issue happening during HTTP message end check. Was also causing issues in chunked message detection.
    Source code(tar.gz)
    Source code(zip)
    gor-v1.3.0_RC5_windows.zip(7.53 MB)
    gor-v1.3.0_RC5.pkg(10.86 MB)
    gor_v1.3.0_RC5_mac.tar.gz(10.86 MB)
    gor-v1.3.0_RC5-1.x86_64.rpm(10.40 MB)
    gor_v1.3.0_RC5_amd64.deb(10.45 MB)
    gor_v1.3.0_RC5_x64.tar.gz(10.44 MB)
  • v1.3.0_RC4(Jun 9, 2021)

    Changes from previous RC

    • Big changes in capture engine to give more performance and stability #916
    • Fixed a bug when GoRepay were including outgoing HTTP request initialized within the app, and having the same port #941
    • Added (again) windows support #940
    • Fix Output HTTP connection leak #927
    Source code(tar.gz)
    Source code(zip)
    gor-1.3.0_RC4_windows.zip(7.52 MB)
    gor-1.3.0_RC4.pkg(10.85 MB)
    gor_1.3.0_RC4_mac.tar.gz(10.85 MB)
    gor-1.3.0_RC4-1.x86_64.rpm(10.40 MB)
    gor_1.3.0_RC4_amd64.deb(10.44 MB)
    gor_1.3.0_RC4_x64.tar.gz(10.44 MB)
  • v1.2.0(Sep 17, 2020)

    • Engine CPU consumption reduced by 60%
    • Implemented M'mapped AF_PACKET for Linux. To enable set --input-raw-engine af_packet. #807
    • Added Kafka TLS configuration flag #800
    • Fix wrong param value updated while --http-set-param #791
    • The engine can now capture more requests with large payload #797
    • Created TCP package #97
    • Fixed minor bugs in HTTP proto parser
    Source code(tar.gz)
    Source code(zip)
    gor-v1.2.0.pkg(11.29 MB)
    gor_v1.2.0_mac.tar.gz(11.29 MB)
    gor-v1.2.0-1.x86_64.rpm(10.88 MB)
    gor_v1.2.0_amd64.deb(10.92 MB)
    gor_v1.2.0_x64.tar.gz(10.92 MB)
  • v1.1.0(Jul 4, 2020)

    • Fix usage of multiple --input-raw plugins, e.g. --input-raw :80 --input-raw :81 --input-raw :82 https://github.com/buger/goreplay/pull/700
    • Allow to set byte size limits in human friedly format: --output-file-size-limit, --output-file-max-size-limit, --copy-buffer-size and --input-raw-buffer-size can now parse inputs from differents bases and data units like: 10mb, 10kb, 100gb, 18t https://github.com/buger/goreplay/pull/754
    • Fix calculation of the output file size when using gzip compression https://github.com/buger/goreplay/pull/777
    • Added --output-tcp-sticky so request/response with same ID sent to the same connection.
    • Better handling of malformed middleware output https://github.com/buger/goreplay/pull/737
    • Fixed race when reading or writing using files https://github.com/buger/goreplay/pull/756 https://github.com/buger/goreplay/pull/764
    • Improved errors verbosity https://github.com/buger/goreplay/issues/703
    • Updated dependencies, including libpcap which should bring more stability and performance
    • Move to Go modules
    • PRO code moved to the main repository under separate license
    Source code(tar.gz)
    Source code(zip)
    goreplay-1.1.0.pkg(11.27 MB)
    gor_1.1.0_mac.tar.gz(11.27 MB)
    goreplay-1.1.0-1.x86_64.rpm(10.81 MB)
    goreplay_1.1.0_amd64.deb(10.86 MB)
    gor_1.1.0_x64.tar.gz(10.86 MB)
  • v1.0.0(Mar 30, 2019)

    Long awaited major release of GoReplay!

    • Added option to specify custom BPF filter (e.g. tcpdump syntax). Can be useful in case of non standard network interfaces like tunneling or SPAN port. Example: --input-raw-bpf-filter 'dst port 80’ https://github.com/buger/goreplay/pull/478
    • Support for reading directly from pcap file, using --input-raw-engine pcap_file --input-raw ./recording.pcap_
    • Added official Docker image docker pull buger/goreplay. Should be run with --network host argument.
    • Added RPM and DEB packages
    • Added support for HTTP proxies. Just set HTTP_PROXY env variable.
    • Added HTTP basic auth filter: --http-basic-auth-filter "^customer[0-9].*. You specify regexp which match userid:passwd string. [https://github.com/buger/goreplay/pull/475]
    • Added option to limit size of output file: when limit is reached, it will exit. Can be used for safety reasons. Example: --output-file-max-size-limit 5gb.
    • Added way to configure output HTTO request queue, which used to hold requests, if all workers are busy. Example: --output-http-queue-len 5000. Default is 1000.
    • Control frequency of --output-http-stats reports using --output-http-stats-msoption. Example: --output-http-stats-ms 5000 (every 5 seconds).
    • Configurable way to set minimum number of HTTP workers, by setting --output-http-workers-min. Can be used in conjunction with --output-http-workers which will act as a max worker count.
    • Added way to dynamically profile GoReplay performance using Golang pprof tools. Example: --http-pprof :8181. It starts web server on given address, and expose special /debug/pprof endpoint with list of reports.
    • Added —input-raw-buffer-size which controls size of the OS buffer (in bytes) which holds packets until they dispatched. Default value depends by system: in Linux around 2MB. If you see big package drop, increase this value.
    • In addition to buffer size, added option to turn pcap immediate mode when packets delivered without buffering. Can help reduce packet drop. Example: --input-raw-immediate-mode.
    • Added way to use standard Golang HTTP client, by adding --output-http-compatibility-mode
    • Snaplen (max number of bytes being read for each packet) now dynamically set based on interface MTU + max header size. In most situations it should reduce package drop, because each packet will consume less space in the buffer. However in some virtualized environments like OpenShift, packet size can be significantly bigger then MTU, so you can disable optimization by setting --input-raw-override-snaplen.
    • Added support for ElasticSearch basic HTTP auth
    • Numerous fixes to improve quality of HTTP packets parsing
    Source code(tar.gz)
    Source code(zip)
    gor_1.0.0_mac.tar.gz(7.08 MB)
    gor_1.0.0_x64.tar.gz(5.45 MB)
    gor-1.0.0-1.x86_64.rpm(7.05 MB)
    gor_1.0.0_amd64.deb(7.08 MB)
  • v1.0-rc2(Feb 23, 2019)

  • v0.16.1(Jun 30, 2017)

    • Add support for TLS connections between --input-tcp and --output-tcp. --input-tcp-secure --input-tcp-certificate ./cert.pem --input-tcp-certificate-key ./key.pem --output-tcp-secure #457
    • Add basic support for old server, like HTTP 0.9 #468 #463 #467
    • GoReplay now exit once finished replaying with --input-file #456
    • Fix prettifier issue, when there is problems with gzip encoding payload #470
    Source code(tar.gz)
    Source code(zip)
    gor_0.16.1_x64.tar.gz(4.61 MB)
    gor_0.16.1_mac.tar.gz(4.34 MB)
  • v0.16.0.2(May 1, 2017)

    Finally a new big release, a lot of changes on all fronts. New features, usability stability, fixes.

    Thank you, everyone, who made it happen!

    It would not be possible without this people @smostovoy @exklamationmark @manjeshnilange @kudos @oivoodoo @sattvik @ylegat @nrwiersma @SophisticaSean

    CHANGELOG

    Major

    • [PRO] Added S3 input and output https://github.com/buger/goreplay/wiki/%5BPRO%5D-Using-S3-for-storing-and-replaying-traffic
    • Added new NodeJS middleware framework https://github.com/buger/goreplay/tree/master/middleware
    • Add Kafka input and output https://github.com/buger/goreplay/wiki/Streaming-from-and-to-Apache-Kafka
    • Improve accuracy of replay by using timestamp of when TCP packet was received by network interface (previously used time of package capture by GoReplay)
    • Using —prettify-http option, you can automatically decode Gzip encoded responses, and de-construct chunked bodies.
    • Now you can enable HTTP response tracking by providing —output-http-track-respose. Previously was available only if middleware is turned on.

    Minor

    • Add new option --input-raw-expire to configure TCP message expiration
    • Filter response if its request was filtered
    • Allow space inside filters syntax: `--http-disallow-header 'Host: www.vertaa.fi' now valid syntax
    • File name pattern now support request id %r, will log each request to separate file, and %t for payload type (0 - request, 1 - response, 2 - replayed response).
    • Add basic SNI support to support HTTP replay to hosts that require SNI, such as Amazon API Gateway. API Gateway)
    • Fix --output-file-size-limit option
    • Do not add port to Host header #383
    • Improve malformed TCP packet handling to avoid panics
    • Fix HTTP timeout for 204 (No content) responses
    • Fix handling of HTTP error codes like 400 or 304.
    • Fixed replay of HTTP PATCH requests
    Source code(tar.gz)
    Source code(zip)
    gor_0.16.0_mac.tar.gz(4.34 MB)
    gor_0.16.0_x64.tar.gz(4.60 MB)
  • v0.15.1(Aug 31, 2016)

  • v0.15.1-beta1(Aug 23, 2016)

  • v0.15.0(Aug 10, 2016)

    • [PRO] Added support for working with binary protocols (thrift/protocol buffers)
    • [PRO] Recording and replaying keep alive TCP sessions
    • Replaying request in proper order when using multiple files #300
    • Validate bodies when Content-Length or Transfer-Encoding found #317
    • Return of ElasticSearch support! #331 #333
    • Added --exit-after option to specify duration after which Gor will exit #336
    • Updated response output file format, now third value is timestamp and forth is latency
    • Properly read body when no Content-Length but there is Connection: close d34c27c
    • Remove --input-http option (too confusing and never really worked) f0acd31
    • Force Go DNS resolver (C one caused crashes for multiple people) 07fa6d9
    • Fix interception on loopback interface when non-local IP used 1ed8691
    • Fix intercepting traffic from virtual interfaces 51860e1
    • Fix 100-continue header when it places not in the end #314
    • Fix relative file names when using --output-file #304
    • Properly cleanup used resources on exit #305
    • Fix connection timeout when using --output-http-timeout c4271ff
    Source code(tar.gz)
    Source code(zip)
    gor_v0.15.0_x64.tar.gz(4.16 MB)
    gor_v0.15.0_mac.tar.gz(3.76 MB)
    gor.exe(7.50 MB)
  • v0.14.1(Jun 10, 2016)

  • v0.14.0(Jun 9, 2016)

    • Windows support!
    • Allow date variables in --output-file names: --output-file %Y-%m-%d-%H.log will create new file each hour #290
    • Support for gzip compressed files, both for input and output #290
    • Replaying from multiple files: --input-file now support file name patterns #290
    • Looping files for replaying indefinitely #290
    • New default --output-file behaviour for writing files in multiple chunks #293
    • Faster buffered file output #290
    • Support for lower-case HTTP headers
    • Added built-in file server for the tutorial: gor file-server :8000
    • Fix bug when gor catch its own --output-http traffic if it replayed to the same port as origin #295
    • --input-raw-realip-header option to injecting header with user real IP #296
    Source code(tar.gz)
    Source code(zip)
    gor.exe(7.13 MB)
    gor_v0.14.0_mac.tar.gz(3.61 MB)
    gor_v0.14.0_x64.tar.gz(4.03 MB)
  • v0.13.0(May 25, 2016)

    • --input-raw now do not track responses by default, you can enable them using --input-raw-track-response #279
    • --output-dummy now renamed to --output-stdout and does not require arguments #282
    • HTTP client response buffer now configurable via --output-http-response-buffer #249
    • Fixed recovery when --output-tcp endpoint goes down https://github.com/buger/gor/commit/0ccace8cad988dfb15ceb725ff3085b5f7c386fd
    • Improved HTTP capture and replaying for requests with multi-packet headers and POST requests #277 #281
    • Fixed libpcap crashing when listening on multiple interfaces https://github.com/buger/gor/commit/98868e81c500579428049944ce47ee50d0ec107b
    • Fixed --input-tcp crashing for payloads > 64kb https://github.com/buger/gor/commit/d40d6c6116c93f5f0f8da768296738cf915821ad
    • Fixed raw socket engine IPv6 support https://github.com/buger/gor/commit/98fe7f19b21eaaffcd0d733a57668f63de5921d7
    Source code(tar.gz)
    Source code(zip)
    gor_0.13.0_mac.tar.gz(3.53 MB)
    gor_0.13.0_x64.tar.gz(3.95 MB)
  • v0.12.1(May 13, 2016)

  • v0.12.0(May 11, 2016)

  • v0.11.2(May 3, 2016)

  • v0.11.1(Apr 29, 2016)

  • v0.11.0(Apr 29, 2016)

    New site and announce of Pro version https://gortool.com

    Release details:

    • License switched to LGPL, and added option for commercial friendly option
    • Default traffic interception engine now libpcap, which significantly increased capture quality and speed
    • Multiple fixes improving stability, speed and quality
    • Go 1.6.1
    Source code(tar.gz)
    Source code(zip)
    gor_0.11.1_x64.tar.gz(2.93 MB)
  • v0.10.1(Sep 3, 2015)

  • 0.10.0(Aug 24, 2015)

  • 0.9.8(Aug 6, 2015)

    • Improve raw-input performance, especially for large payloads #170
    • Use all cores by default #170
    • Improve code styling, thanks to golint. #171
    • Added --http-disallow-header option #173
    • Do not override host if it explicitly specified by--http-host #182
    • Add option to keep original Host headers #188
    • Configurable http timeout via --output-http-timeout #183
    • Fix http response buffer overflow #185
    • Fix trailing null byte while communicating between intput-tcp and output-tcp #176
    Source code(tar.gz)
    Source code(zip)
    gor_x86.tar.gz(1.94 MB)
    gor_x64.tar.gz(2.01 MB)
  • 0.9.7(Jul 10, 2015)

  • 0.9.6(Jul 9, 2015)

    • New faster HTTP client #160.
    • Filtering and rewriting support extended to all output plugins #160
    • Fixed requests with Expect: 100-continue #158
    • Added request param filtering/limiting, see --html-set-param and --html-param-limiter #165
    • Added option to disallow certain url patterns --http-disallow-url #165
    • You now can specify multiple --http-allow-url and --http-disallow-url filters #165
    • Use percent based numbers for header/param limiters, instead of fractions (old syntax supported) #165
    • header/param limiters should not reject urls if param/header not found. To mimic this functionality use second --http-allow-header filter. #165
    • Improved flag names, made it more consistent and predictable. Old ones also supported but deprecated. #165
    • Improved handling of HTTP1.0 and proxy support #163
    • Changed internal TCP data format #164
    • Better code organization #165
    Source code(tar.gz)
    Source code(zip)
    gor_x86.tar.gz(1.93 MB)
    gor_x64.tar.gz(2.01 MB)
Owner
Leonid Bugaev
👉 👉 👉 👉 👉 👉 👉 👉 👉 👉 👉
Leonid Bugaev
A Java library for capturing, crafting, and sending packets.

Japanese Logos Pcap4J Pcap4J is a Java library for capturing, crafting and sending packets. Pcap4J wraps a native packet capture library (libpcap, Win

Kaito Yamada 898 Sep 1, 2021
Pcap editing and replay tools for *NIX and Windows - Users please download source from

Tcpreplay Tcpreplay is a suite of GPLv3 licensed utilities for UNIX (and Win32 under Cygwin) operating systems for editing and replaying network traff

AppNeta, Inc. 806 Sep 8, 2021
Chaos engineering tool for simulating real-world distributed system failures

Proxy for simulating real-world distributed system failures to improve resilience in your applications. Introduction Muxy is a proxy that mucks with y

Matt Fellows 771 Sep 13, 2021
A Linux packet crafting tool.

Pig Pig (which can be understood as Packet intruder generator) is a Linux packet crafting tool. You can use Pig to test your IDS/IPS among other stuff

Rafael Santiago 399 Aug 26, 2021
A Java library that implements a ByteChannel interface over SSLEngine, enabling easy-to-use (socket-like) TLS for Java applications.

TLS Channel TLS Channel is a library that implements a ByteChannel interface over a TLS (Transport Layer Security) connection. It delegates all crypto

Mariano Barrios 119 Sep 8, 2021
Simulating shitty network connections so you can build better systems.

Comcast Testing distributed systems under hard failures like network partitions and instance termination is critical, but it's also important we test

Tyler Treat 7.6k Sep 15, 2021
jRT measures the response time of a java application to socket-based requests

jRT Version: 0.0.1 jRT is a instrumentation tool that logs and records networking I/O operations "response times" (applicaion response time if be corr

null 44 Feb 5, 2020
API gateway for REST and SOAP written in Java.

API gateway for REST and SOAP written in Java.

predic8 GmbH 357 Sep 10, 2021
ssh, scp and sftp for java

sshj - SSHv2 library for Java To get started, have a look at one of the examples. Hopefully you will find the API pleasant to work with :) Getting SSH

Jeroen van Erp 2k Sep 12, 2021
TCP/UDP client/server library for Java, based on Kryo

KryoNet can be downloaded on the releases page. Please use the KryoNet discussion group for support. Overview KryoNet is a Java library that provides

Esoteric Software 1.6k Sep 10, 2021
Apache Dubbo is a high-performance, java based, open source RPC framework.

Apache Dubbo Project Apache Dubbo is a high-performance, Java-based open-source RPC framework. Please visit official site for quick start and document

The Apache Software Foundation 36.1k Sep 11, 2021
A High Performance Network ( TCP/IP ) Library

Chronicle-Network About A High Performance Network library Purpose This library is designed to be lower latency and support higher throughputs by empl

Chronicle Software : Open Source 185 Sep 18, 2021
Asynchronous Http and WebSocket Client library for Java

Async Http Client Follow @AsyncHttpClient on Twitter. The AsyncHttpClient (AHC) library allows Java applications to easily execute HTTP requests and a

AsyncHttpClient 5.8k Sep 17, 2021
Reactive stubs for gRPC

What is reactive-grpc? Reactive gRPC is a suite of libraries for using gRPC with Reactive Streams programming libraries. Using a protocol buffers comp

Salesforce 669 Aug 30, 2021
Socket.IO server implemented on Java. Realtime java framework

Netty-socketio Overview This project is an open-source Java implementation of Socket.IO server. Based on Netty server framework. Checkout Demo project

Nikita Koksharov 5.4k Sep 13, 2021
TCP/IP packet demultiplexer. Download from:

TCPFLOW 1.5.0 Downloads directory: http://digitalcorpora.org/downloads/tcpflow/ Installation Most common GNU/Linux distributions ship tcpflow in their

Simson L. Garfinkel 1.3k Sep 12, 2021
Magician is an asynchronous non-blocking network protocol analysis package, supports TCP, UDP protocol, built-in Http, WebSocket decoder

An asynchronous non-blocking network protocol analysis package Project Description Magician is an asynchronous non-blocking network protocol analysis

贝克街的天才 88 Sep 11, 2021
IoT Platform, Device management, data collection, processing and visualization, multi protocol, rule engine, netty mqtt client

GIoT GIoT: GIoT是一个开源的IoT平台,支持设备管理、物模型,产品、设备管理、规则引擎、多种存储、多sink、多协议(http、mqtt、tcp,自定义协议)、多租户管理等等,提供插件化开发 Documentation Quick Start Module -> giot-starte

gerry 28 Aug 7, 2021
A networking framework that evolves with your application

ServiceTalk ServiceTalk is a JVM network application framework with APIs tailored to specific protocols (e.g. HTTP/1.x, HTTP/2.x, etc…) and supports m

Apple 694 Sep 10, 2021