OpenGrok is a fast and usable source code search and cross reference engine, written in Java

Overview

Copyright (c) 2006, 2020 Oracle and/or its affiliates. All rights reserved.

OpenGrok - a wicked fast source browser

Github actions build Coverage status SonarQube status Total alerts License

1. Introduction

OpenGrok is a fast and usable source code search and cross reference engine, written in Java. It helps you search, cross-reference and navigate your source tree. It can understand various program file formats and version control histories of many source code management systems.

Official page of the project is on: http://opengrok.github.com/OpenGrok/

2. OpenGrok install and setup

See https://github.com/oracle/opengrok/wiki/How-to-setup-OpenGrok

2. 1. Updating

OpenGrok uses semantic versioning and the version components further indicate more details about updating to newer version. The version scheme is x.y.z and change in any component is interpreted as:

  • x - major backwards incompatible update
  • y - full clean reindex of your repositories is needed (e. g. index format has changed)
  • z - redeploy web application

3. Information for developers

See https://github.com/oracle/opengrok/wiki/Developer-intro and https://github.com/oracle/opengrok/wiki/Developers

4. Authors

The project has been originally conceived in Sun Microsystems by Chandan B.N.

For full list of contributors see https://github.com/oracle/opengrok/graphs/contributors

5. Contact us

There are Slack channels on https://opengrok.slack.com/

6. Run as container

You can run OpenGrok as a Docker container as described here.

Issues
  • perforce problem when doing OpenGrok index update

    perforce problem when doing OpenGrok index update

    Hi All,

    I am looking for OpenGrok index update documentation, both manual and automated, please advise. Ideally I'd like to have it configured each time source files are updated, or on a scheduled basis. If not supported, manual option will do too as long as it doesn't rebuild index from scratch every time.

    Thanks, YM

    bug question 
    opened by ymnick 75
  • can't select project after indexing

    can't select project after indexing

    Hey, after indexing using java -Djava.util.logging.config.file=/opengrok/etc/logging.properties -jar /opengrok/dist/lib/opengrok.jar -c /usr/local/bin/ctags -s /opengrok/src -d /opengrok/data -H -P -S -G -W /opengrok/etc/configuration.xml -U http://localhost:8080/source I can't select projects to search on image

    also, I have attached my configration.xml configuration.zip

    Thank you for your help

    UI question 
    opened by chkp-baselz 63
  • Feature/deferred ops

    Feature/deferred ops

    Hello,

    Please consider for integration this patch with some improvements in error handling and changes to reduce the time interval where the Lucene database is inconsistent with the xref file data and to speed up deletions for issue #1908.

    1. Add PendingFileCompleter and use in IndexDatabase to queue up file deletions and file renamings to be executed during the Lucene 2-phase commit. IndexDatabase is changed to write to in-progress files (i.e., with an "._org_opengrok" extension) while xref-ing and to queue up the moves to final names (without the pending extension) during the 2-phase commit. This minimizes the time interval of inconsistency between the xref data and the committed Lucene data.
    1. Do not run Lucene commit() on each file deletion, but instead once as the last step of IndexDatabase.update() so that Lucene and PendingFileCompleter can be well-coordinated.
    1. Do not swallow IndexWriter exceptions for prepareCommit(), commit(), or close() in either the IndexDatabase update() or optimize() methods.
    2. Support NativeFSLockFactory, and support the --lock switch via the OpenGrok script (on|off|native|simple with the default still off, and on as an alias for simple). Configuration is revised to support the new lock mode, with the existing boolean usingLuceneLocking preserved but @Deprecated to be retired at some future point — i.e., after a sufficient upgrade time interval is deemed to have past.
    3. Ensure a meaningful non-zero exit code if a non-fatal error occurs in the OpenGrok script.

    Thank you.

    opened by idodeclare 54
  • java.lang.OutOfMemoryError

    java.lang.OutOfMemoryError

    Environment: OpenGrook 0.12.1, Linux, Java 7 64-bit, >JAVA_OPTS="-Xmx16g -Xms16g"<

    Scanning about 300 projects, small , to medium size, total 20GB of all source code Scan results with "java.lang.OutOfMemoryError: Requested array size exceeds VM limit"

    opened by andycoolware 40
  • setting configuration property via API is not propagated to all threads

    setting configuration property via API is not propagated to all threads

    Today I think I hit a race condition when doing the parallel indexing and using the sync.py script (mostly from the wiki).

    My flow is following:

    1. Delete all unwanted projects with projadm.py
    2. Download all sources from the SCM
    3. Add all projects with projadm.py (no index at this step)
    4. Generate groups and publish them to the webapp
    5. Index the sources using sync.py script with mirroring.

    I'm using 10 repositories, some of them take longer time to index but most of them can finish in almost the same time. I'm using the number of workers 4.

    When the sync.py script is finished, with no errors, the page itself displays only 9 (sometimes 8) projects most of the time, with one of them missing randomly. After looking in the webapp configuration the project itself is not marked as indexed (therefore not visible).

    However, looking at the indexer logs, there is a 10 times a mention that the uri projects/{p}/indexed has been fired and return 204.

    -bash-4.1$ cat /opengrok/log/**/opengrok0.0.log | grep "indexed HTTP/1.1" | wc -l
    10
    

    Also looking at the tcpdump communication (posted below), I can see 10 calls for the indexed url.

    I couldn't dig any deeper at this point, but there's something wrong going on.

    Sync-config.yml:

    commands:
    - command:
      - http://localhost:8150/source/api/v1/messages
      - POST
      - cssClass: info
        duration: PT2H
        tags: ['%PROJECT%']
        text: resync + reindex in progress
    - command: [ /opengrok/bin/mirror.py, -U, 'http://localhost:8150/source', -c, /opengrok/etc/mirror-config.yml ]
    - command: [
        /opengrok/bin/reindex-project.py,
        -j,
        '/usr/java/jdk1.8.0_60/bin/java',
        -J=-d64,
        '-J=-XX:-UseGCOverheadLimit',
        -J=-Xmx12g,
        -J=-server,
        -J=-Dorg.opengrok.indexer.history.git=/opt/git/git283/bin/git,
        --jar,
        /opengrok/lib/opengrok.jar,
        -t,
        /opengrok/etc/logging.properties.template,
        -p,
        '%PROJ%',
        -d,
        /opengrok/log/%PROJECT%,
        -P,
        '%PROJECT%',
        --uri,
        'http://localhost:8150/source',
        --,
        -U,
        'http://localhost:8150/source',
        --renamedHistory,
        'off',
        -r,
        dirbased,
        --memory,
        '256',
        -c,
        /opt/universal_ctags/bin/ctags,
        -i,
        'Assets.webtest',
        -P,
        -H,
        '%PROJECT%',
      ]
      env: {LC_ALL: en_US.UTF-8}
    - command: ['http://localhost:8150/source/api/v1/messages?tag=%PROJECT%', DELETE, '']
    cleanup:
      command: ['http://localhost:8150/source/api/v1/messages?tag=%PROJECT%', DELETE, '']
    
    

    Tcpdump:

    [[email protected]]# tcpdump -s 0 -A -i lo -v -l -n  port 8150 | egrep -i "POST /|GET /|PUT /|DELETE /|Host:"
    tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes
    [email protected]@oPOST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    [email protected]@pPOST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    [email protected]@zPOST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    [email protected]@.POST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ..C'..C'GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ..C3..C3GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ..CZ..CZGET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ..Cm..CmGET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ..C...C.GET /source/api/v1/projects/api.fuel/repositories HTTP/1.1
    Host: localhost:8150
    ..C...C.GET /source/api/v1/repositories/type?repository=%2Fapi.fuel HTTP/1.1
    Host: localhost:8150
    ..C...C.GET /source/api/v1/projects/api.apitest/repositories HTTP/1.1
    Host: localhost:8150
    ..D...D.GET /source/api/v1/repositories/type?repository=%2Fapi.apitest HTTP/1.1
    Host: localhost:8150
    ..D%..D$GET /source/api/v1/projects/api.avails/repositories HTTP/1.1
    Host: localhost:8150
    ..D)..D)GET /source/api/v1/repositories/type?repository=%2Fapi.avails HTTP/1.1
    Host: localhost:8150
    ..D7..D7GET /source/api/v1/projects/api.analytics/repositories HTTP/1.1
    Host: localhost:8150
    ..D;..D;GET /source/api/v1/repositories/type?repository=%2Fapi.analytics HTTP/1.1
    Host: localhost:8150
    ..G...G.GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ..HI..HIGET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ..H...H.GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ..J...J.GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ..O...O.PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ..Pt..PqPUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ..P...P.PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ..Sc..S`PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ..{E..{DPUT /source/api/v1/projects/api.apitest/indexed HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/projects/api.avails/indexed HTTP/1.1
    Host: localhost:8150
    [email protected]{VPUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ...v...rDELETE /source/api/v1/messages?tag=api.apitest HTTP/1.1
    Host: localhost:8150
    ...~...~POST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ...V...VGET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/projects/api.fuel_js/repositories HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/repositories/type?repository=%2Fapi.fuel_js HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ...T...TDELETE /source/api/v1/messages?tag=api.avails HTTP/1.1
    Host: localhost:8150
    ..._..._POST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ...a...aGET /source/api/v1/projects/api.navajo/repositories HTTP/1.1
    Host: localhost:8150
    ...g...fGET /source/api/v1/repositories/type?repository=%2Fapi.navajo HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ...'....PUT /source/api/v1/projects/api.fuel_js/indexed HTTP/1.1
    Host: localhost:8150
    .......6PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ........DELETE /source/api/v1/messages?tag=api.fuel_js HTTP/1.1
    Host: localhost:8150
    ........POST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ...e...eGET /source/api/v1/projects/api.orion/repositories HTTP/1.1
    Host: localhost:8150
    ...k...kGET /source/api/v1/repositories/type?repository=%2Fapi.orion HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ...,...#PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ...>...=PUT /source/api/v1/projects/api.analytics/indexed HTTP/1.1
    Host: localhost:8150
    ...m...hPUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ...j...fDELETE /source/api/v1/messages?tag=api.analytics HTTP/1.1
    Host: localhost:8150
    ...r...rPOST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/projects/api.orion-gateway/repositories HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/repositories/type?repository=%2Fapi.orion-gateway HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ...t...kPUT /source/api/v1/projects/api.orion/indexed HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    .......TPUT /source/api/v1/projects/api.orion-gateway/indexed HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ...-...%DELETE /source/api/v1/messages?tag=api.orion-gateway HTTP/1.1
    Host: localhost:8150
    ...9...1POST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    [email protected]@GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ........DELETE /source/api/v1/messages?tag=api.orion HTTP/1.1
    Host: localhost:8150
    ........POST /source/api/v1/messages HTTP/1.1
    Host: localhost:8150
    ...m...mGET /source/api/v1/projects/api.orionapitest/repositories HTTP/1.1
    Host: localhost:8150
    ...u...qGET /source/api/v1/repositories/type?repository=%2Fapi.orionapitest HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration/sourceRoot HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/projects/api.oxy/repositories HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/repositories/type?repository=%2Fapi.oxy HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ........GET /source/api/v1/configuration HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ...q...mPUT /source/api/v1/configuration/projectsEnabled HTTP/1.1
    Host: localhost:8150
    ...k....PUT /source/api/v1/projects/api.orionapitest/indexed HTTP/1.1
    Host: localhost:8150
    .. ... .PUT /source/api/v1/projects/api.fuel/indexed HTTP/1.1
    Host: localhost:8150
    ..&.....PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ..)...).DELETE /source/api/v1/messages?tag=api.orionapitest HTTP/1.1
    Host: localhost:8150
    ..N...N.PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ..O[..O[DELETE /source/api/v1/messages?tag=api.fuel HTTP/1.1
    Host: localhost:8150
    ..O...O.PUT /source/api/v1/projects/api.oxy/indexed HTTP/1.1
    Host: localhost:8150
    ..`#..O.PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ..a<..a<DELETE /source/api/v1/messages?tag=api.oxy HTTP/1.1
    Host: localhost:8150
    ...{...yPUT /source/api/v1/projects/api.navajo/indexed HTTP/1.1
    Host: localhost:8150
    ........PUT /source/api/v1/system/refresh HTTP/1.1
    Host: localhost:8150
    ...t...tDELETE /source/api/v1/messages?tag=api.navajo HTTP/1.1
    Host: localhost:8150
    [[email protected]]# 
    
    bug 
    opened by tulinkry 37
  • Failed to get xref file

    Failed to get xref file

    Hey :)

    After indexing using this command java -Djava.util.logging.config.file=/opengrok/etc/logging.properties -jar /opengrok/dist/lib/opengrok.jar --ctags /usr/local/bin/ctags -s /opengrok/src -d /opengrok/data -H -P -S -G -W /opengrok/etc/configuration.xml -U http://localhost:8080/source/ some files can't be open although it could be read and exist in src folder image

    Can't find anything special in log, but when checked found this for almost all projects in catalina.out SEVERE [ForkJoinPool-331-worker-19] org.opengrok.suggest.Suggester.lambda$getInitRunnable$1 Could not initialize suggester data for aftershock java.io.FileNotFoundException: /opengrok/data/suggester/aftershock/defs.wfst (Permission denied) at java.base/java.io.FileOutputStream.open0(Native Method) at java.base/java.io.FileOutputStream.open(FileOutputStream.java:298) at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:237) at java.base/java.io.FileOutputStream.<init>(FileOutputStream.java:187) at org.opengrok.suggest.SuggesterProjectData.store(SuggesterProjectData.java:273) at org.opengrok.suggest.SuggesterProjectData.build(SuggesterProjectData.java:250) at org.opengrok.suggest.SuggesterProjectData.init(SuggesterProjectData.java:154) at org.opengrok.suggest.Suggester.lambda$getInitRunnable$1(Suggester.java:229) at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

    Appreciate your help Basel

    enhancement indexer 
    opened by chkp-baselz 36
  • History is enabled in index.sh, but no git history in webapp when running indexer without -S

    History is enabled in index.sh, but no git history in webapp when running indexer without -S

    I'm running in a container based on a docker image built from opengrok/docker, and an altered index.sh that currently looks like this:

    #!/bin/bash
    
    LOCKFILE=/var/run/opengrok-indexer
    URI="http://localhost:8080"
    OPS=${INDEXER_FLAGS:='-H -P -G'}
    
    if [ -f "$LOCKFILE" ]; then
            date +"%F %T Indexer still locked, skipping indexing"
            exit 1
    fi
    
    touch $LOCKFILE
    
    date +"%F %T Indexing starting"
    opengrok-indexer \
        -J=-Xmx8g \
        -J=-server \
        -a /opengrok/lib/opengrok.jar -- \
        -i hugeMatlabFile.mat \
        -m 256 \
        -v \
        --repository /opengrok/src/master \
        -s /opengrok/src \
        -d /opengrok/data \
        --remote on \
        -W /opengrok/etc/configuration.xml \
        -U "$URI" \
        $OPS \
        $INDEXER_OPT "[email protected]"
    date +"%F %T Indexing finished"
    

    The index forms ok, and seems fine to a cursory examination, but the history link doesn't work, and there's no list of "current version" with last commit on the web app front page, which I get with the 1.5 year old version, if I run it on the same files.


    The only severe I found in the log is a handful of these:

    WARNING: CTags parsing problem: java.io.IOException: Stream closed at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170) at java.io.BufferedInputStream.read(BufferedInputStream.java:336) at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284) at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326) at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178) at java.io.InputStreamReader.read(InputStreamReader.java:184) at java.io.BufferedReader.fill(BufferedReader.java:161) at java.io.BufferedReader.readLine(BufferedReader.java:324) at java.io.BufferedReader.readLine(BufferedReader.java:389) at org.opengrok.indexer.analysis.Ctags.readTags(Ctags.java:547) at org.opengrok.indexer.analysis.Ctags.lambda$doCtags$2(Ctags.java:455) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

    Mar 16, 2020 11:31:08 PM org.opengrok.indexer.analysis.Ctags readTags SEVERE: CTag reader cycle was interrupted!


    Is this a settings issue? Did I put in the wrong flag somewhere? I mean, as far as I understand it, the -H means history is on, no?

    Or is it the severe?

    Either way, please advise... and please excuse my being a pain.

    docker indexer question 
    opened by Ymoise 35
  • OpenGrok don't create History for Perforce repository.

    OpenGrok don't create History for Perforce repository.

    Hello.

    I wan't use OpenGrok for create repository browser on Tomcat server on my Ubuntu. Indexer works fine - but the problem I don't know how integrate it with Perforce.

    I run this command: $JAVA -Xms2G -Xmx16G -Dorg.opensolaris.opengrok.history.Perforce=/usr/bin/p4 -jar "${OpenGrokJar}" -r on -H -m 2048 -c "$CTAGS" -w "myProject" -s "$VIEW" -d "$DB_DIR50" $IGNORE_PATTERNS $VERBOSE_PROGRES -z 1000 -t 4 -P -vvv -W "$CONF_FILE" -a on -O on

    I was run this command : p4 dir /$VIEW and it return that this location is under P4 Client.

    In log file I have lines: INFO: Generating history cache for all repositories ... Sep 27, 2018 10:20:14 AM org.opensolaris.opengrok.history.HistoryGuru createCacheReal INFO: Creating historycache for 0 repositories INFO: Done historycache for all repositories (took 6 ms)

    Please, could you point me how integrate OpenGrok with Perforce - step by step.

    question 
    opened by AndrzejPolska 34
  • Indexer option to run historycache only

    Indexer option to run historycache only

    Is your feature request related to a problem? Please describe. Let's write yes. I am reindexing all my company Git repos (30k) with latest OG 1.5.11 / Tomcat 9 / Java 11 / RHEL 8.3. Historycache generation is extremely long...

    Describe the solution you'd like I would like when reindexing: 1a run indexer without historycache 1b run another indexer process in parallel with historycache only that's to say skipping file indexing since this is done in a short time by process of step 1a 2 each night run indexer without historycache 3 when historycache started in step 1b finishes ok, then and only then, update job run nightly in step 2 adding historycache too

    Describe alternatives you've considered Optimizing the historycache generation? Issue to be opened further to your answer to this one. But in all case, because of the size of our code sources, historycache process has always been very long.

    Additional context Some of our repos have very long history, with many tags and also many files. This does not help generating quickly the history.

    enhancement indexer 
    opened by ChristopheBordieu 33
  • how to have multiple Perforce repositories as projects

    how to have multiple Perforce repositories as projects

    This is a question. I have a source tree, /disk/src, and under that are top-level directories for multiple (currently 12) projects from multiple source repositories. I am running opengrok indexer with -s /disk/src -d /disk/data -H -P -S -G. What the output from the console shows, and the log file, includes:

    11:08:10 INFO: Scanning for repositories...
    11:08:11 INFO: Done scanning for repositories, found 1 repositories (took 825 ms)
    11:08:11 INFO: Generating history cache for all repositories ...
    11:08:11 INFO: Creating historycache for 1 repositories
    11:08:11 INFO: Creating historycache for /disk/src (PerforceRepository) without renamed file handling
    11:41:42 INFO: Done historycache for /data/opengrok/src (took 0:33:31)
    11:41:42 INFO: Done historycache for all repositories (took 0:33:31)
    11:41:42 INFO: Done...
    11:41:42 INFO: Starting indexing
    11:41:43 INFO: Waiting for the executors to finish
    11:41:43 INFO: Creating historycache for /disk/src (PerforceRepository) without renamed file handling
    11:41:43 INFO: Creating historycache for /disk/src (PerforceRepository) without renamed file handling
    11:41:43 INFO: Creating historycache for /disk/src (PerforceRepository) without renamed file handling
    11:41:43 INFO: Creating historycache for /disk/src (PerforceRepository) without renamed file handling
    

    It seems to be creating projects properly, when I view results in the U/I later, but. Are the log messages just misleading, or am I doing something wrong?

    (side note, only some of the projects are Perforce, some are git, it seems to always say /disk/src (PerforceRepository) which is in a way the most concerning part of the incorrect log messages...)

    question 
    opened by cross 32
  • parallelize directory creation for history cache

    parallelize directory creation for history cache

    As mentioned in PR #3636 the directory cache creation in FileHistoryCache#createDirectoriesForFiles() can be parallelized and made more efficient by avoiding mkdirs() with the same arguments.

    enhancement indexer 
    opened by vladak 0
  • present tags as list in history API endpoint

    present tags as list in history API endpoint

    As mentioned in https://github.com/oracle/opengrok/pull/3636#discussion_r652917370 the tags can be stored as a list, not string. The history API endpoint would benefit.

    API enhancement 
    opened by vladak 0
  • Unable to get any Suggestions

    Unable to get any Suggestions

    Hello all,

    I am unable to get any Suggestions from OpenGrok. Could someone please list out a bunch of reasons why this might be happening?

    Thank you!

    question suggester 
    opened by adarsh200496 5
  • what is tomcat.servlet.context.path for in the pom file?

    what is tomcat.servlet.context.path for in the pom file?

    What is purpose of the parameter

    <tomcat.servlet.context.path>/source</tomcat.servlet.context.path>

    in the pom file?

    Could it be used in the the target

    <build>
    	<finalName>source</finalName>
    

    making the change

    	<finalName>${tomcat.servlet.context.path}</finalName>
    

    ?

    build enhancement 
    opened by cosmoJFH 5
  • Opengrok crashes and complains about missing configuration file

    Opengrok crashes and complains about missing configuration file

    Describe the bug Opengrok crashes with this error in web browser:

    There was an error! The source root path has not been configured! Please configure your webapp.

    Unable to determine source root path. Missing configuration?

    java.io.FileNotFoundException: Unable to determine source root path. Missing configuration?

    configuration.xml exists under $ROOT/etc/ and is configured in tomcat webapp.

    To Reproduce Steps to reproduce the behavior:

    Expected behavior A clear and concise description of what you expected to happen.

    Screenshots image

    Additional context Add any other context about the problem here.

    question 
    opened by devidebyzero 2
  • Error viewing any file in a specific project with Docker + version 1.7.7/1.7.8

    Error viewing any file in a specific project with Docker + version 1.7.7/1.7.8

    Describe the bug Running OG 1.7.7 in Docker (and same in 1.7.8 verified just now), one of my git repos under a project fails to render any file that I click on with the following dump:

    There was an error!
    Walk failure.
    
    org.eclipse.jgit.errors.RevWalkException: Walk failure.
    	at org.eclipse.jgit.revwalk.RevWalk.nextForIterator(RevWalk.java:1380)
    	at org.eclipse.jgit.revwalk.RevWalk.iterator(RevWalk.java:1402)
    	at org.opengrok.indexer.history.GitRepository.getHistory(GitRepository.java:520)
    	at org.opengrok.indexer.history.GitRepository.getHistory(GitRepository.java:461)
    	at org.opengrok.indexer.history.GitRepository.getHistory(GitRepository.java:456)
    	at org.opengrok.indexer.history.FileHistoryCache.get(FileHistoryCache.java:608)
    	at org.opengrok.indexer.history.HistoryGuru.getHistory(HistoryGuru.java:264)
    	at org.opengrok.web.PageConfig.getLastRevFromHistory(PageConfig.java:1296)
    	at org.opengrok.web.PageConfig.getLatestRevision(PageConfig.java:1288)
    	at org.apache.jsp.list_jsp._jspService(list_jsp.java:286)
    	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:71)
    	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:770)
    	at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:467)
    	at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:379)
    	at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:327)
    	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:770)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:223)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    	at org.opengrok.web.CookieFilter.doFilter(CookieFilter.java:50)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    	at org.opengrok.web.AuthorizationFilter.doFilter(AuthorizationFilter.java:95)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    	at org.opengrok.web.StatisticsFilter.doFilter(StatisticsFilter.java:67)
    	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
    	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
    	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
    	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543)
    	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
    	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:353)
    	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
    	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:870)
    	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1696)
    	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    	at java.base/java.lang.Thread.run(Thread.java:829)
    Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing commit d803f74934a46848b448b71420dbe21a9cc39a67
    	at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:137)
    	at org.eclipse.jgit.revwalk.RevWalk.getCachedBytes(RevWalk.java:961)
    	at org.eclipse.jgit.revwalk.RevCommit.parseHeaders(RevCommit.java:126)
    	at org.eclipse.jgit.revwalk.TreeRevFilter.include(TreeRevFilter.java:113)
    	at org.eclipse.jgit.revwalk.PendingGenerator.next(PendingGenerator.java:108)
    	at org.eclipse.jgit.revwalk.BlockRevQueue.<init>(BlockRevQueue.java:40)
    	at org.eclipse.jgit.revwalk.FIFORevQueue.<init>(FIFORevQueue.java:37)
    	at org.eclipse.jgit.revwalk.StartGenerator.next(StartGenerator.java:133)
    	at org.eclipse.jgit.revwalk.RevWalk.next(RevWalk.java:443)
    	at org.eclipse.jgit.revwalk.RevWalk.nextForIterator(RevWalk.java:1378)
    	... 45 more
    

    I don't see the referenced missing commit in the project at all (via git show ), but I also don't see any obvious historical issue running local git log commands, etc, in that git project. This problem opengrok project has about a dozen git projects under it and two of the git repos have this error (unique missing commit hash in the two separate projects of course, but same error signature). A separate similarly indexed project that happens to point at another branch in the same git repo is fine.

    bug webapp 
    opened by jwehler 6
  • There is no data in web after restart tomcat or docker whithout  no code change and reindex

    There is no data in web after restart tomcat or docker whithout no code change and reindex

    There is no data in web after restart tomcat or docker, but data and config is no change, so can i just upload configuration.xml ?

    There was an error! The source root path has not been configured! Please configure your webapp.

    Unable to determine source root path. Missing configuration?

    java.io.FileNotFoundException: Unable to determine source root path. Missing configuration?

    docker question 
    opened by menggya 1
  • Rejuvenate log levels

    Rejuvenate log levels

    Here's a reissue of https://github.com/oracle/opengrok/pull/2978 which is closed due to inactivity and conflicts. We have run our logging tool on the last version of the project and solved all conflicts. We appreciate any feedback and are willing to make further changes if you wish to incorporate our PR into your project.

    Settings

    We have several analysis settings. We can vary these settings and rerun if you desire. The settings we are using in this pull request are:

    • Treat CONFIG, WARNING, and SEVERE levels as a category and not a traditional level, i.e., our tool ignores these log levels.
    • Never lower the logging level of logging statements within catch blocks.
    • Never lower the logging level of logging statements within if statements.
    • Never lower the logging level of logging statements containing certain (important) keywords.
    • Never raise the logging level of logging statements without particular keywords in their messages.
    • Avoid log wrapping by disregarding logging statements contained in if statements mentioning log levels.
    • Only consistently transform the level of logging statements appearing in overriding methods.
    • Adjust transformations if their transformation distances are over maximum transformation distance.

    The greatest number of commits from HEAD evaluated: 5000. The head at the time of analysis was: a31748b478960db5458c5383e8a9282fa6037af6

    OCA

    I signed and emailed the OCA to Oracle. I am also providing a written acceptance of the OCA line:

    I am signing on behalf of myself as an individual and no other person or entity, including my employer, has or will have rights with respect my contributions.

    opened by saledouble 4
  • CSS framework?

    CSS framework?

    The UI needs some revamp as it is quite outdated. I think CSS framework would make it easier for us (just engineers without any designers) to provide a decent-looking site. However, which one to use?

    I've used Bootstrap a little and was satisfied with it and it seems to be the most popular option. UIkit seems to be used in OCI docs.

    The list of frameworks: https://github.com/troxler/awesome-css-frameworks

    Related issues: https://github.com/oracle/opengrok/issues/3215 https://github.com/oracle/opengrok/issues/2406

    UI question 
    opened by ahornace 1
  • Suggester and search do not work properly for words written in hiragana (and probably other CJK alphabets)

    Suggester and search do not work properly for words written in hiragana (and probably other CJK alphabets)

    While testing https://github.com/oracle/opengrok/pull/1391 I've found out that OpenGrok has some issues with hiragana letters.

    I've added three words in hiragana to the index:

    あめ
    ありがと
    あした
    

    Typing does not make suggester to suggest anything meaningful. Searching for the whole word splits it into multiple characters (e.g. +full:あ +full:し +full:た) and does not provide any results. Adding * seems to be a workaround for search and suggester but distorts the suggester output: Screen Shot 2021-05-23 at 18 42 43

    bug suggester webapp 
    opened by ahornace 0
Releases(1.7.10)
Owner
Oracle
Open Source at Oracle
Oracle
⚡️Lightning-fast linter for .env files. Written in Rust 🦀

⚡️ Lightning-fast linter for .env files. Written in Rust ?? Dotenv-linter can check / fix / compare .env files for problems that may cause the applica

null 933 Jun 9, 2021
Inria 1.1k Jun 16, 2021
A tool to help eliminate NullPointerExceptions (NPEs) in your Java code with low build-time overhead

NullAway: Fast Annotation-Based Null Checking for Java NullAway is a tool to help eliminate NullPointerExceptions (NPEs) in your Java code. To use Nul

Uber Open Source 3k Jun 7, 2021
Sourcetrail - free and open-source interactive source explorer

Sourcetrail Sourcetrail is a free and open-source cross-platform source explorer that helps you get productive on unfamiliar source code. Windows: Lin

Coati Software 11.5k Jun 18, 2021
:coffee: SonarSource Static Analyzer for Java Code Quality and Security

Code Quality and Security for Java This SonarSource project is a code analyzer for Java projects. Information about the analysis of Java features is a

SonarSource 787 Jun 16, 2021
Astra: a Java tool for analysing and refactoring Java source code

What is Astra? Astra is a Java tool for analysing and refactoring Java source code. For example: "References to type A should instead reference type B

Alfa 32 Jun 8, 2021
A static analyzer for Java, C, C++, and Objective-C

Infer Infer is a static analysis tool for Java, C++, Objective-C, and C. Infer is written in OCaml. Installation Read our Getting Started page for det

Facebook 12.4k Jun 13, 2021
Continuous Inspection

SonarQube Continuous Inspection SonarQube provides the capability to not only show health of an application but also to highlight issues newly introdu

SonarSource 5.9k Jun 16, 2021
Tackle Data-intensive Validity Analyzer

Tackle-DiVA (Data-intensive Validity Analyzer) Tackle-DiVA is a command-line tool for data-centric application analysis. It imports a set of target ap

Konveyor 4 May 18, 2021
An extensible multilanguage static code analyzer.

PMD About PMD is a source code analyzer. It finds common programming flaws like unused variables, empty catch blocks, unnecessary object creation, and

PMD 3.4k Jun 14, 2021
SpotBugs is FindBugs' successor. A tool for static analysis to look for bugs in Java code.

SpotBugs is the spiritual successor of FindBugs, carrying on from the point where it left off with support of its community. SpotBugs is licensed unde

null 2.3k Jun 8, 2021
Catch common Java mistakes as compile-time errors

Error Prone Error Prone is a static analysis tool for Java that catches common programming mistakes at compile-time. public class ShortSet { public

Google 5.6k Jun 14, 2021
Your Software. Your Structures. Your Rules.

jQAssistant Master Repository We splitted jQAssistant in multiple single repositories to be able to build a better and more flexible build an release

null 159 May 30, 2021