Apache Lucene.NET

Overview

Apache Lucene.NET

Nuget Azure DevOps builds (master) GitHub

Full-text search for .NET

Apache Lucene.NET is a .NET full-text search engine framework, a C# port of the popular Apache Lucene project. Apache Lucene.NET is not a complete application, but rather a code library and API that can easily be used to add search capabilities to applications.

The Apache Lucene.NET web site is at: http://lucenenet.apache.org

Supported Frameworks

Lucene.NET 3.0.3

  • .NET Framework 4.0
  • .NET Framework 3.5

Lucene.NET 4.8.0

Status

Latest Stable Version: Lucene.NET 3.0.3

Working toward Lucene.NET 4.8.0 (currently in BETA)

Download

Lucene.NET 3.0.3

Core Library

NuGet version

PM> Install-Package Lucene.Net
All Packages

Lucene.NET 4.8.0

Core Library

NuGet version

PM> Install-Package Lucene.Net -Pre
All Packages

Documentation

We now have some preliminary documentation for Lucene.NET 4.8.0 on the Lucene.NET Website.

The API is similar to Java Lucene 4.8.0, which you may also find helpful to review.

NOTE: We are working on fixing issues with the documentation, but could use more help since it is a massive project. See #206.

Legacy Versions

Demos & Tools

There are several demos implemented as simple console applications that can be copied and pasted into Visual Studio or compiled on the command line in the Lucene.Net.Demo project.

There is also a dotnet command line tool available on NuGet. It contains all of the demos as well as tools maintaining your Lucene.NET index, featuring operations such as splitting, merging, listing segment info, fixing, deleting segments, upgrading, etc. Always be sure to back up your index before running any commands against it!

dotnet tool install lucene-cli -g --version 4.8.0-beta00012

NOTE: The version of the CLI you install should match the version of Lucene.NET you use.

Once installed, you can explore the commands and options that are available by entering the command lucene.

lucene-cli Documentation

How to Contribute

We love getting contributions! Read our Contribution Guide or read on for ways that you can help.

Join Mailing Lists

How to Join Mailing Lists

Ask a Question

If you have a general how-to question or need help from the Lucene.NET community, please subscribe to the user mailing list by sending an email to [email protected] and then follow the instructions to verify your email address. Note that you only need to subscribe once.

After you have subscribed to the mailing list, email your message to [email protected].

Alternatively, you can get help via StackOverflow's active community.

Please do not submit general how-to questions to GitHub, use GitHub for bug reports and tasks only.

Report a Bug

To report a bug, please use the GitHub issue tracker.

NOTE: In the past, the Lucene.NET project used the JIRA issue tracker, which has now been deprecated. However, we are keeping it active for tracking legacy issues. Please submit any new issues to GitHub.

Start a Discussion

To start a development discussion regarding technical features of Lucene.NET, please email the dev mailing list by sending an email to [email protected] and then follow the instructions to verify your email address. Note that you only need to subscribe once.

After you have subscribed to the mailing list, email your message to [email protected].

Submit a Pull Request

Before you start working on a pull request, please read our Contributing guide.

Building and Testing

Command Line

Building on the Command Line is currently only supported on Windows.

Prerequisites
  1. Powershell 3.0 or higher (see this question to check your Powershell version)
  2. .NET 5.0 SDK or higher
  3. .NET Framework 4.8 Developer Pack
Execution

To build the source, clone or download and unzip the repository. From the repository root, execute:

> build [options]
Build Options
Short Long Description Example
‑config ‑‑Configuration The build configuration ("Release" or "Debug"). build ‑‑Configuration:Debug
‑mp ‑‑MaximumParallelJobs The maximum number of parallel jobs to run during testing. If not supplied, the default is 8. build ‑t ‑mp:10
‑pv ‑‑PackageVersion The NuGet package version. If not supplied, will use the version from the Version.proj file. build ‑pv:4.8.0‑beta00001
‑t ‑‑Test Runs the tests after building. Note that testing typically takes around 40 minutes with 8 parallel jobs. build ‑t
‑v ‑‑Version The assembly file version. If not supplied, will use the PackageVersion (excluding any pre-release tag). build ‑pv:4.8.0‑beta00001 ‑v:4.8.0

NuGet packages are output by the build to the /release/NuGetPackages/ directory. Test results (if applicable) are output to the /release/TestResults/ directory.

You can setup Visual Studio to read the NuGet packages like any NuGet feed by following these steps:

  1. In Visual Studio, right-click the solution in Solution Explorer, and choose "Manage NuGet Packages for Solution"
  2. Click the gear icon next to the Package sources dropdown.
  3. Click the + icon (for add)
  4. Give the source a name such as Lucene.Net Local Packages
  5. Click the ... button next to the Source field, and choose the /src/release/NuGetPackages folder on your local system.
  6. Click Ok

Then all you need to do is choose the Lucene.Net Local Packages feed from the dropdown (in the NuGet Package Manager) and you can search for, install, and update the NuGet packages just as you can with any Internet-based feed.

Visual Studio

Prerequisites

  1. Visual Studio 2019 or higher
  2. .NET 5.0 SDK or higher
  3. .NET Framework 4.8 Developer Pack

Execution

  1. Open Lucene.Net.sln in Visual Studio.
  2. Choose the target framework to test by opening build/TestTargetFramework.props and uncommenting the corresponding <TargetFramework> (and commenting all others).
  3. Build a project or the entire solution, and wait for Visual Studio to discover the tests - this may take several minutes.
  4. Run or debug the tests in Test Explorer, optionally using the desired filters.

NOTE: When running tests in Visual Studio, be sure to set the default processor architecture to 64 bit to avoid running out of virtual memory on some tests.

Azure DevOps

We have setup our azure-pipelines.yml file with logical defaults so anyone with an Azure DevOps account can build Lucene.NET and run the tests with minimal effort. Even a free Azure DevOps account will work, but tests will run much faster if the account is setup as public, which enables up to 10 parallel jobs to run simultaneously.

Prerequisites

  1. An Azure DevOps account.
  2. A fork of this repository either on GitHub or Azure DevOps. The rest of these instructions assume a GitHub fork.

Execution

If you don't already have a pipeline set up:
  1. Create an Azure DevOps organization. If you already have one that you wish to use, you may skip this step.
  2. Create an Azure DevOps project. We recommend naming the project Lucene.NET. Note that if you are using a free Azure DevOps account, you should choose to make the project public in order to enable 10 parallel jobs. If you make the project private, you will only get 1 parallel job. Also, if disabling features, make sure to leave Pipelines enabled.
  3. Create an Azure DevOps pipeline.
    • Click on "Pipelines" from the left menu.
    • Click the "Create Pipeline" or "New Pipeline" button, depending on whether any pipelines already exist.
    • Select GitHub as the location to find the YAML file.
    • Select the fork of this repository you created in "Prerequisites". Note that if this is a new Azure DevOps account you may need to setup extra permissions to access your GitHub account.
    • Next a "Review your YAML" page is presented showing the contents of azure-pipelines.yml. There is documentation near the top of the file indicating the variables that can be setup to enable additional options, but note that the default configuration will automatically run the build and all of the tests.
    • Click the "Run" button at the top right of the page.
If you already have a pipeline set up:
  1. Click on "Pipelines" from the left menu.
  2. Select the pipeline you wish to run.
  3. Click the "Queue" button on the upper right.
  4. (Optional) Select the branch and override any variables in the pipeline for this run.
  5. Click the "Run" button.

Note that after the build is complete, the nuget artifact contains .nupkg files which may be downloaded to your local machine where you can setup a local folder to act as a NuGet feed.

It is also possible to add an Azure DevOps feed id to a new variable named ArtifactFeedID, but we are getting mixed results due to permission issues.

Issues
  • Migrating Lucene.Net to .NET Core

    Migrating Lucene.Net to .NET Core

    Prerequisites

    Building Lucene.Net.Portable.sln

    This will build Lucene.NET for .NET Standard and .NET 4.5.1.

    Instructions

    There is a build script, Build.ps1 that performs build steps. For more information, execute Get-Help Build.ps1 -Full in Powershell.

    Building

    1. Go to repository root
    2. Execute: Build.ps1 -CreatePackages

    Testing

    • Execute all tests: Build.ps1 -RunTests
    • Execute specific tests: Build.ps1 -RunTests -ProjectsToTest @("Lucene.Net.Tests", "Lucene.Net.Tests.Facet")
    • Execute Manually:
      1. cd to any of the testing projects
      2. dotnet test --where="Category!=LongRunningTest && Category != DtdProcessingTest && Category != HasTimeout"

    Building Lucene.Net.sln

    This will build Lucene.NET using the old projects for .NET 4.5.1. Hopefully Lucene.Net.sln can be removed in the future because the .NET Core projects compile for .NET 4.5.1 already.

    • Visual Studio
      1. Open solution in Visual Studio 2015
      2. Restore NuGet packages (right-click on solution, restore NuGet packages)
      3. Build solution
    • Command Line
      1. Open a Visual Studio 2015 developer command prompt
      2. Restore packages using NuGet v3
      3. Execute msbuild Lucene.Net.sln

    Notes

    These are some explicit things to highlight about changes that were made in order to get it to build for .NET Standard 1.5

    Compiler preprocessors

    Since .NET Standard is a subset of the full .NET Framework, there are some features that are not supported, so they had to be compiled out until support for them was enabled. The following preprocessors were added to the code:

    • FEATURE_SERIALIZABLE - Binary serialization (ie. [Serializable]) is not supported in .NET Core.
    • FEATURE_CLONEABLE - ICloneable interface isn't being brought back until .NET Standard 2.0 (Spring 2017)
    • FEATURE_DTD_PROCESSING - DtdProcessing.Parse and XmlResolver isn't being brought back until .NET Standard 2.0 (Spring 2017). Issues #4376, #10322
    • FEATURE_THREADPOOL_UNSAFEQUEUEWORKITEM - Will be added back in .NET Standard 2.0. Issue #11633
    • FEATURE_TASKMERGESCHEDULER - ConcurrentMergeScheduler has a lot of Thread management that is not supported in .NET Standard until 2.0. This enables exclusive usage of TaskMergeScheduler.
    • FEATURE_EMBEDDED_RESOURCE - Embedding resources using .xproj/project.json results in different resource names than using .csproj. This was added to support existing logic in Lucene.Net.sln.
    • NET451 - Using xproj, this compiler processor is automatically defined but had to be added to the csproj because support for setting CultureInfo.CurrentCulture did not exist until .NET 4.6.

    ICU and Build Architecture

    NUnit

    • Added [HasTimeout] attribute because NUnit on .NET Core does not have [Timeout]. As a result, we could have tests that run forever. See nunit/nunit#1658. These tests are skipped during a test run using Build.ps1's $ExcludeTestCategoriesNetCore parameter
    • Currently, tests run against netcoreapp1.0 because Visual Studio Test Explorer fails to show tests when they are compiled against multiple frameworks. nunit/dotnet-test-nunit#37. It would work in the command-line though, so we have to decide whether or not to only run tests from command-line or have an experience in Visual Studio. (We can manually edit the project.json to run tests against net451, too, when we are executing dotnet test from command line. Not sure how ideal this would be.)
    • I had to migrate the tests to NUnit 3 because dotnet-test-nunit uses NUnit 3
    • At first, I tried to migrate the tests to xUnit, but there are a lot of issues. Mainly that xUnit tests run in parallel and with the static properties in Lucene.NET tests and test framework, it made it hard to break those dependencies and run the tests reliably.

    Test Suite Status

    Project | Failed | Passed | Done? ---|---|---|--- Lucene.Net.Tests | 25 | 2539 | Yes | Lucene.Net.Tests.Analysis.Common | 0 | 1440 | Yes | Lucene.Net.Tests.Analysis.Stempel | 0 | 16 | Yes | Lucene.Net.Tests.Classification | 0 | 9 | Yes | Lucene.Net.Tests.Codecs | 0 | 448 | Yes | Lucene.Net.Tests.Expressions | 0 | 94 | Yes | Lucene.Net.Tests.Facet | 0 | 155 | Yes | Lucene.Net.Tests.Grouping | 0 | 12 | Yes | Lucene.Net.Tests.Join | 0 | 27 | Yes | Lucene.Net.Tests.Memory | 0 | 10 | Yes | Lucene.Net.Tests.Misc | 0 | 42 | Yes | Lucene.Net.Tests.Queries | 0 | 96 | Yes | Lucene.Net.Tests.QueryParser | 0 | 302 | Yes | Lucene.Net.Tests.Suggest | 0 | 137 | Yes |

    opened by conniey 66
  • Website & API Doc site generator using DocFx script

    Website & API Doc site generator using DocFx script

    This is a PR to build both a new website and the API documentation using DocFx.

    There are several changes:

    • build script for both the docs and the website
    • updates to the JavaDocToMarkdownConverter (which is the processor to convert the lucene Java docs to a usable md version for docfx ... this is an ongoing update to deal with all of the quirks)
      • Whenever this is run, it will fix/update many of the md converted doc files
    • website files for docfx

    To test the website you can run: websites/site/site.ps1 which will build the site and start a webserver at http://localhost:8080, any changes made to the site just stop the script (ctrl + c) and re-run it and it will do an incremental build. To just build the website for deployment, run websites/site/site.ps1 -ServeDocs 1 -Clean 1 which will clean all temp files and compile the website to a static website at websites/site/_site.

    To test the docs you can run: websites/apidocs/docs.ps1 which will build the site and start a webserver at http://localhost:8080, any changes made to the docs just stop the script (ctrl + c) and re-run it and it will do an incremental build. To just build the docs for deployment, run websites/apidocs/docs.ps1 -ServeDocs 1 -Clean 1 which will clean all temp files and compile the website to a static website at websites/apidocs/_site.

    (In both cases, the build operation takes a few minutes!)

    • The new website is already live at https://lucenenet.apache.org/
    • A demo of the new docs site is here until we get this hosted properly: https://lucenenetdocs.azurewebsites.net/

    Website tasks to complete:

    • [ ] Ensure branding policies are adhered to http://www.apache.org/foundation/marks/pmcs
    • [ ] Add Build server info to the Contributing section, see https://teamcity.jetbrains.com/project.html?projectId=LuceneNet_PortableBuilds&tab=projectOverview
    • [ ] Migrate any of the very important info from the wiki to the getting started section (don't need all of it, but there's probably some critical info there that should be front and center) https://cwiki.apache.org/confluence/display/LUCENENET/Lucene.Net
    • [x] Copy/condense the important info from the GitHub readme to the Contributing section https://github.com/apache/lucenenet/blob/master/README.md
    • [ ] Copy/condense the contributing info from the GitHub readme to the Contributing section https://github.com/apache/lucenenet/blob/master/CONTRIBUTING.md
    • [x] Probably add a small section on version 3.0.3 and 2.9.4 with links
    • [ ] Add remaining effort on project to website to be very clear and obvious, some info is here: https://github.com/apache/lucenenet/blob/master/CONTRIBUTING.md#porting-work---up-for-grabs and other info is here https://lists.apache.org/thread.html/[email protected]%3Cdev.lucenenet.apache.org%3E, but there's also the ICU project that apparently needs work too
    • [ ] (Determine if this is absolutely required!) Update download page and include the CGI/mirror/signed zip thing, see 3rd section of this comment https://github.com/apache/lucenenet/pull/206#issuecomment-393800264 , source for that is here http://svn.apache.org/repos/asf/lucene.net/site/trunk/
    • [x] Figure out how to host the site, there is some info here https://lists.apache.org/thread.html/[email protected]%3Cdev.lucenenet.apache.org%3E (it's currently using Apache CMS). Since the output of this is a static html/js site it should work (but need to sort out the CGI download page within this build), there are 3 hosting options for apache sites: http://www.apache.org/dev/project-site.html ... seems like gitpubsub is the easiest and most flexible to use.

    Docs tasks to complete:

    • [x] Style custom token replacement such as @lucene.experimental and @lucene.internal
    • [x] Customized Table of Contents so the nav bar isn't one giant list (we can do this with post build)
    • [ ] Ensure view port widths are optimized (might already be fixed with latest doc fx)
    • [ ] Fix all invalid cref issues listed in build
    • [ ] Part of the build process should be to fetch the 4.8 java source from https://github.com/apache/lucene-solr/releases/tag/releases%2Flucene-solr%2F4.8.0 to run the JavaToMarkdownConverter program against that prior to running the docs generator (instead of manually doing this)
    • [ ] Remove the TestFramework namespace from the docs (easy to do in the docfx json file i think)
    • [ ] Version the Docs URLs, for example, the current docs are here http://lucenenet.apache.org/docs/3.0.3/Index.html, the 4.8.0 ones should be here: http://lucenenet.apache.org/docs/4.8.0/Index.html , this will need to be part of the docs build script to pass along a version number, see 2nd section in this comment https://github.com/apache/lucenenet/pull/206#issuecomment-393800264
    • [ ] Figure out how to host the docs site

    Additional tasks (nice to have):

    • [ ] Update the build server to generate all docs + website + zips, deploy to staging, have a vote, etc... see comment: https://github.com/apache/lucenenet/pull/206#issuecomment-393834720
    • [ ] Differentiate between .NET Standard and .NET Framework. This is supposed to be possible with docfx though I'm unsure it will be the ideal way we want yet
    • [ ] Investigate overlapping namespaces with DocFx, might be that we need to ask on the DocFx GH site, see https://github.com/apache/lucenenet/pull/206#issuecomment-357590055
    opened by Shazwazza 50
  • RFC: LUCENENET-565: Porting of Lucene.Net.Replicator

    RFC: LUCENENET-565: Porting of Lucene.Net.Replicator

    Commit is for Review with comments about original Java Source for assistance.

    • [x] RevisionFile.Size - Please change to Length to .NETify
    • [x] IReplicator & IRevision: Please leave the "I" prefix in the actual name of the interfaces, but change the names of the files back to the original "Replicator" and "Revision". It is easier to maintain if the files are in the same order as they are in Java.
    • [x] Please change IReadOnlyCollection, IReadOnlyDictionary, etc to ICollection, IDictionary, etc.
    • [x] Please rename (the class only, not the file) IndexInputInputStream to IndexInputStream (since in .NET InputStream is just a Stream).
    • [x] ~~Please add the project.json and PROJECTNAME.project.json files to the projects (we use these for NuGet dependencies as well as for specifying NuGet package info). See the other projects for examples.~~ See note
    • [x] Please arrange the code so it is in a similar order as it was in Java. In cases where you need to order it differently (such as de-nesting types), add a "LUCENENET specific" comment to indicate it diverges from Java.
    • [x] ~~Please ensure it will compile and tests will pass on .NET Core (we use the Lucene.Net.Portable.sln file to do these tests).~~ See note
    • [x] Replace BinaryFormatter with Json
    • [x] Use DataInputStream and DataOutputStream instead of DataInput and DataOutput.

    Note: Async -> Sync: There is quite some Async to Sync bridging, ideally I would wan't to provide an Async interface around underlying Async operations, but it has to be done carefully and is therefore somewhat time consuming (I tried but if broke everything)... It wouldn't break compatibility though so it should be something that can be added in a later version.

    Note: Missing .NET Core project files because of the current situation with .NET Core/Standard project files as discussed in https://issues.apache.org/jira/browse/LUCENENET-565

    Additional tasks before merge:

    • [x] Remove all //Java: comments, they are there for review purposes only.
    • [x] Move EnumrableExtensions to Lucene.Support? or Lucene.Util?
    • [x] Rethink HttpClientBase and HttpReplicator
    • [x] Rethink ReplicatorService, abstractions and AspNetCore implementation.
    • [x] Document with Examples of integration with all parts covered.
    opened by jeme 30
  • Updates docs build

    Updates docs build

    • Fixes build of csproj doc tools to not include the strong name signing
    • Fixes some namespaces issues in the JavaDocToMarkdownConverter
    • Re-executes the JavaDocToMarkdownConverter and commits updated files (incuding whitespaces so on next run there's no changes)
    • Fixes UI auto-expanding quirk with docfx and the docs (hamburger menu)
    • Fixes some links from the docs home page and removes the ICU to be completed note

    I have upload the result of this to our temporary docs site https://lucenenetdocs.azurewebsites.net/

    @NightOwl888 just regarding your comments here https://github.com/apache/lucenenet/pull/206#issuecomment-519378752

    home page/package names

    Can you elaborate a bit more on this:

    It would probably be easier to understand if we updated the names on the home page to reflect the package names

    ... I've moved the Kuromoji and SmartCn headings outside of the big Analysis heading since they are separate packages and linked to them properly from the home page, is this the type of thing you mean? I also fixed the ICU link on the home page.

    (there's currently still an issue with docfx when there is overlapping namespaces between packages, i haven't yet researched into how to fix this but i can, the docfx team are very responsive)

    build/build times

    A full clean build takes about 20 mins on my machine so should be ok for the build server. On the build server you'd want to run the powershell:

    ./websites/apidocs/docs.ps1 0 1
    

    which is shorthand for

    ./websites/apidocs/docs.ps1 --ServeDocs 0 --Clean 1
    

    The output website is in ./websites/apidocs/_site

    versioning

    I know that in the JavaDocToMarkdownConverter there's a TODO for passing in a tab/version which is for the method RepoLinkReplacer ... but, this method is looking for links in files like overview.md with this syntax src-html, but as far as i can see there is only 2x places in all of the source that contains these types of links which is in the Lucene.Net/overview.md file which is supposed to link to some demos. For this method it would prob just be easier to fix this file, unless there are more inline links i'm unsure of.

    Apart from that is there another area where we need to have the version number/tag injected in places?

    update: just noticed a version link here https://lucenenetdocs.azurewebsites.net/index.html#reference-documents ... so we'd need to pass a version into the build for that one, do you know of others?

    hosting

    The docs are currently just hosted on my azure subscription which is also why it has that temporary dns name. I'm fine to leave it there for any length of time but we should look at getting these hosted properly, it's just static files so nothing special.

    opened by Shazwazza 27
  • Error using Lucene.Net.Facet 4.8.0-beta00005  with Xamarin.iOS

    Error using Lucene.Net.Facet 4.8.0-beta00005 with Xamarin.iOS

    I'm using Lucene.Net.Facet 4.8.0-beta00005 in a big Xamarin project.

    With Xamarin.Android and Xamarin.UWP it's all right.

    But With Xamarin.iOS on device (Ipad), i'm receiving this error:

    Attempting to JIT compile method 'Lucene.Net.Support.LurchTable2:InternalInsert> (int,Lucene.Net.Facet.Taxonomy.FacetLabel,int&,Lucene.Net.Support.LurchTable`2/Add2Info&)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.

    at Lucene.Net.Support.LurchTable2[TKey,TValue].Insert[T] (TKey key, T& value) <0x2570f48 + 0x000e0> in <063e095c95d945a4ace32ab83d1227eb#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at (wrapper unknown) System.Object.gsharedvt_in() at Lucene.Net.Support.LurchTable2[TKey,TValue].AddOrUpdate (TKey key, TValue addValue, Lucene.Net.Support.KeyValueUpdate2[TKey,TValue] fnUpdate) <0x232824c + 0x0013b> in <063e095c95d945a4ace32ab83d1227eb#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at Lucene.Net.Facet.Taxonomy.LRUHashMap2[TKey,TValue].Put (TKey key, TValue value) <0x2c487f8 + 0x0015b> in <79d3a7b905954d0993025c09c5d087ce#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at Lucene.Net.Facet.Taxonomy.Directory.DirectoryTaxonomyReader.GetOrdinal (Lucene.Net.Facet.Taxonomy.FacetLabel cp) <0x2c51970 + 0x0019b> in <79d3a7b905954d0993025c09c5d087ce#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at Lucene.Net.Facet.Taxonomy.Int32TaxonomyFacets.GetTopChildren (System.Int32 topN, System.String dim, System.String[] path) <0x2c481dc + 0x0008f> in <79d3a7b905954d0993025c09c5d087ce#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at Login.MyMB.Lucene.Client.LuceneArticoliSearcher.GetListaArticoloXRicercaAvanzataConRicercaSemplice (System.Collections.Generic.List1[T] listParametri) <0x224add0 + 0x001bb> in <8f49891e0f0546e185aba7424d294ef7#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at Login.MyMB.Lucene.Client.LuceneArticoliSearcher.GetListaArticoloConRicercaSemplice (System.Collections.Generic.List1[T] listParametri) <0x224afbc + 0x0009f> in <8f49891e0f0546e185aba7424d294ef7#2ae0fea9ea4eacaef83bf2e9713bb8ea>:0 at MyMB.Forms.RicercaLucene.RicercaArticoloLucene.GetListaArticoliXRicercaSemplice (Login.MyMB.Interface.IAmbiente ambiente, Login.MyMB.Lucene.Client.LuceneArticoliSearcher las, System.Collections.Generic.List`1[T] ListParametri, System.Boolean isAbilitataRicercaBarcode) <0xe47fc0 + 0x000e7> in :0 ...............................

    At the link https://docs.microsoft.com/it-it/xamarin/ios/internals/limitations , I found the problem cause (I suppose...):

    Value types as Dictionary Keys Using a value type as a Dictionary key is problematic, as the default Dictionary constructor attempts to use EqualityComparer.Default. EqualityComparer.Default, in turn, attempts to use Reflection to instantiate a new type which implements the IEqualityComparer interface. This works for reference types (as the reflection+create a new type step is skipped), but for value types it crashes and burns rather quickly once you attempt to use it on the device. Workaround: Manually implement the IEqualityComparer interface in a new type and provide an instance of that type to the Dictionary (IEqualityComparer) constructor.

    So, what can I do? Thank you in advance, Enrico Caltran +393357485560 [email protected]

    JIRA link - [LUCENENET-602] created by enycaltran Lucene.Net.Facet 
    opened by clambertus 27
  • Remove Debugging.AssertsEnabled

    Remove Debugging.AssertsEnabled

    Doing some benchmarking here, just saw this strange one on FixedBitSet.cs

    @NightOwl888 do you mind if I replace this with a conditional compilation flag?

    image

    opened by theolivenbaum 23
  • IndexOutOfRangeException when searching

    IndexOutOfRangeException when searching

    Hello, I'm getting an IndexOutOfRangeException when searching in some cases. It's happening maybe 10% of the time so I'm unsure what is causing this. See below for the search code and stack trace. I feel like it might be something with the query but its more of a guess. Any guidance on this would be very much appreciated.

    var sort = new Sort(new SortField(null, SortFieldType.DOC)); return _searcher.Search(Query, _reader.NumDocs, sort);

    FATAL Update Data Set System.IndexOutOfRangeException: Index was outside the bounds of the array. at Lucene.Net.Store.ByteArrayDataInput.ReadVInt32() at Lucene.Net.Codecs.BlockTreeTermsReader.FieldReader.IntersectEnum.Frame.NextLeaf() at Lucene.Net.Codecs.BlockTreeTermsReader.FieldReader.IntersectEnum.Next() at Lucene.Net.Search.TermCollectingRewrite1.CollectTerms(IndexReader reader, MultiTermQuery query, TermCollector collector) at Lucene.Net.Search.ConstantScoreAutoRewrite.Rewrite(IndexReader reader, MultiTermQuery query) at Lucene.Net.Join.ToParentBlockJoinQuery.Rewrite(IndexReader reader) at Lucene.Net.Search.IndexSearcher.Rewrite(Query original) at Lucene.Net.Search.IndexSearcher.CreateNormalizedWeight(Query query) at Lucene.Net.Search.IndexSearcher.Search(Query query, Int32 n, Sort sort)`

    investigation is:bug 
    opened by jregnier 22
  • Docs - Build/Deploy Automation

    Docs - Build/Deploy Automation

    Building the API documentation files for a given release is currently a manual process and also involves manually updating the websites.

    As part of our build/deploy pipeline we need to automate as much of this as possible.

    • [x] document the docs building procedure
    • [x] document the website updating procedure
    • [x] create new 4.8.0-beta00008 api docs release
    • [x] automate more of the docs building procedure including passing in version numbers to populate global variables and branch targets
      • ~Need to figure out how to use global variables in the md with docfx~
    • [x] update website with new links
    • [x] ensure the website has correct download links
    docs 
    opened by Shazwazza 21
  • Best way to setup dev machine for LuceneNet website edits

    Best way to setup dev machine for LuceneNet website edits

    @Shazwazza

    Hi Shannon,

    I'd like to make the following small addition to the home page of the Lucene.net website give developers more information about the Lucene 4.8 version which will hopefully give them the confidence to give it a try. Specifically I'd like to add something similar to:

    Latest Version - Lucene 4.8 Beta

    • The beta version is extremely stable
    • Has more than 7000+ passing unit tests
    • Integrates well with .Net 5.0
    • Supports NetStandard 2.0 and NetStandard 2.1
    • Supports with .Net Framework 4.5.1+
    • Some developers already use it in production environments

    It's a pretty easy add but I need to first configure my dev machine for it. Initially I just downloaded the a zip file of the master at https://github.com/apache/lucenenet-site/ to check it out. Looks like a static website, IIS not required, and I didn't see any Visual Studio project files in the folder structure. So I just double clicked the index.html file to view it in the browser but it doesn't quite render properly. On my machine it looks like this below (Notice the missing blue banner):

    mine

    Whereas the actual site looks like this:

    expected

    So I may be mistaken and it may not be designed to run from the desktop like this. Any pointers on how I can setup a dev environment locally to do some changes to the index.html file?

    docs is:idea is:question 
    opened by rclabo 19
  • Add Contributor's guide

    Add Contributor's guide

    @synhershko wrote A contributor's guide on the mailing list.

    Seems like the kind of thing that would make sense to put in the repository. :)

    The purpose of this PR is to:

    • Add the contributor's guide e-mail from Itamar to the repository
    • Add both of the existing markdown documents as solution items to the Visual Studio solution.
    opened by SeanKilleen 17
  • Fetching a docvalue from the index

    Fetching a docvalue from the index

    I am trying to figure out a way to fetch a binary docvalue from the index but I can't find anything online, Any suggestions on how to achieve this? Any help with this would be much appreciated. Thanks

    opened by jregnier 1
  • NLP Support (OpenNLP)

    NLP Support (OpenNLP)

    I don't know if this is an issue or a discussion yet, but it seems logical to document this somewhere in case we make it to release with gaps in support for NLP.

    First of all, Lucene 4.8.0 didn't support Apache OpenNLP, it supported Apache UIMA. So, we picked a newer Lucene version (8.2.0) and did what it did, choosing OpenNLP instead of UIMA (which is seemingly now part of the OpenNLP package). This was primarily because there were no options available for UIMA in .NET, but some options for supporting OpenNLP and it didn't make sense to do the work to support UIMA in .NET that would only last through 1 Lucene.NET major release.

    Options for NLP Support in .NET

    Option Issues Notes
    Port OpenNLP from version 1.9.1 tag to .NET
    • The project is large and would take a lot of effort to port and maintain.
    Use AlexPoint/OpenNlp from NuGet
    • API has been refactored significantly from OpenNLP and would take a high-level analysis to use the new API
    • It isn't clear what version of OpenNLP this is as the version number doesn't seem to track the one in Java, but it is probably long before 1.9.1 and seems to be missing features Lucene uses
    • Currently only supports .NET Framework 4.5+
    Use Standford NLP.NET
    • The API is significantly different from OpenNLP and it would take a high-level analysis to determine whether it has the features we need
    • It is an IKVM port, which currently only supports .NET Framework 3.5
    • Its GNU2 license is too restrictive to use in an Apache project (we can depend on, but not import code)
    There is a project called Tweet NLP that extends it and seems to supply much of the functionality Lucene uses
    Use AboditNLP from GitHub
    • A high-level analysis is required to determine if it supports the functionality Lucene uses.
    • Closed-source, only demos and the NuGet package are available.
    Targets .NET Framework 4.7.2, .NET Standard 2.0, and .NET Standard 2.1.
    Use CherubNLP from NuGet
    • Would require a high-level analysis to determine if it supports the functionality Lucene uses
    Targets .NET Standard 2.0.
    Use OpenNLP.NET from GitHub
    • It is an IKVM port, which currently only supports .NET Framework 3.5
    This is the option we currently use. Someone created a strong-named package named OpenNLP.NET.Signed. It would be preferable to get the original package owner to strong-name, but I suppose that would mean incrementing to at least version 1.9.1.1, or upgrading to a newer version of OpenNLP.

    There are some other options, but the above list seem to be the most "official" ones. However, there are currently no options for .NET Core/.NET 5+ support of OpenNLP with the same API as OpenNLP 1.9.1.

    IKVM

    Unfortunately, while IKVM has been a reasonable go-to way to quickly support Java-based apps in the past, it has been abandoned by its main contributor in 2017 and has no .NET Core/NET Standard support.

    There is an effort to get it working on .NET Core named ikvm-revived (to which I have contributed) but it seems to have been stalled for about a year and, as of the date of this writing, there isn't even a pre-release on NuGet. There is some debate whether they should support .NET Framework, but if they didn't we would still be able to target the current OpenNLP.NET version on .NET Framework.

    See NuGet Repository?

    Alternatives to IKVM

    There was an announcement on the Microsoft Blog about .NET 5 supporting interoperability with Java, but it isn't clear what they meant by that.

    https://devblogs.microsoft.com/dotnet/announcing-net-5-0-preview-1/#comment-4932

    In fact, others are mentioning in the comments they cannot use NLP on .NET Core and are hoping to resolve that in .NET 5.

    I have searched, but cannot find any examples anywhere of how .NET 5 supports Java interop, but if it does that would probably be a better path forward than IKVM for NLP support. However, it sounds as if this feature was punted from the official .NET 5 release.

    Current Support for NLP in Lucene.NET

    Since we are depending on the IKVM-based OpenNLP.NET project, our current support is limited to .NET Framework 4.5.1+.

    We do have some minor issues (namely lack of InternalsVisibleTo support) due to the fact that the library is not strong-named, but these are internal. Time will tell if lack of strong-naming is going to be an issue for end users, but ideally to get strong naming we should contribute to OpenNLP.NET rather than using the strong-named clone named OpenNLP.NET.Signed.

    Supporting NLP in .NET Core/.NET 5+

    Most options for supporting NLP on .NET Core would require some work to put into play, and it isn't clear how much work is involved to analyze this at a high level. It also isn't clear how big the demand for this functionality will be.

    While we could make an effort to change dependencies, it would be sensible to create a new assembly named after the new dependency (in the src/dotnet folder) so it is clear what it depends on and leave the existing Lucene.Net.Analysis.OpenNLP project as-is.

    Another option is just to wait to see whether ikvm-revieved releases a .NET Core targeted package on NuGet and then support it when they finally do.

    NOTE: If we bring back support for native .NET Collation in Lucene.Net.Analysis.Common, it is possible that its SortKeys would not be portable between .NET Framework and .NET Core/.NET 5+ (see Caveats and Comparisons). If we don't have .NET Core/.NET 5 support for Lucene.Net.OpenNLP, that collator option could cause some issues if indexing can only be done on .NET Framework, but searching is done on .NET Core or .NET 5. However, we have a collator in Lucene.Net.ICU that is stable across .NET target frameworks that could be used instead in that scenario.

    Finally, I will add that while we would prefer not to hold up the release of Lucene.NET 4.8.0 to wait on .NET Core/.NET 5+ support of NLP, if some companies are willing to provide some resources/funding because they need to have either an OpenNLP or an IKVM option on .NET Core/.NET 5+, we could probably make that happen.

    dependency pri:low 
    opened by NightOwl888 1
  • docs/website: Change highlight.js Theme

    docs/website: Change highlight.js Theme

    On both our main website and the API docs site, we should change the theme of the code highlighter. There are 2 issues we hope to solve:

    • Make the code look more like it does in Visual Studio.
    • Make the inline code blocks more visible. Currently, they look like disabled text.

    It really strains your eyes to look at the current theme too long.

    There is a blog post that specifically outlines the steps how to change the theme with DocFx:

    https://www.cazzulino.com/customize-docfx.html

    One theme I thought would be okay was the VS theme here, but the one from the blog post (which I haven't previewed) might be closer to what we are looking for. But as long as the text looks less faded than it does now, it would be an improvement.

    Note that the themes of DocFx are already exported to the /websites/apidocs/Templates/DefaultTemplateNoAssets folder so they can be copied over to the "live" folders and modified easily.

    Project Infrastructure docs good-first-issue is:enhancement pri:normal up-for-grabs 
    opened by NightOwl888 3
  • Task: Optimize and Document End-to-End Demo and Benchmark Experience

    Task: Optimize and Document End-to-End Demo and Benchmark Experience

    Deliverables for this task

    Goal

    The goal is to make this information easy to find, easy to understand, and test it to make sure the examples work as documented.


    Based on the Lucene demo document here:

    https://github.com/apache/lucenenet/blob/Lucene.Net_4_8_0_beta00014/src/Lucene.Net.Demo/overview.md

    it is clear that there are gaps in our knowledge of how these demos were meant to be delivered to Java developers and how we can provide an optimal experience in .NET.

    What Lucene does

    • The demo "package" is shipped as compiled code and hosted on Maven
    • The documentation seems to assume that you know how to get from there to "adjust your classpath" to being able to run from the command line, without giving much detail about how
    • It is not clear whether the command line is run against a compiled .jar file or if they are building from source

    How we "ported" it

    • We don't ship the Lucene.Net.Demo package on NuGet
    • Since it is not possible to "run" a .NET Standard assembly, the demos were compiled into a .NET Core 3.1 console application named lucene-cli.
    • We compile the code from the Lucene.Net.Demo project into lucene-cli and ship that on NuGet. The demos can be run directly from the console, provided the right input options are provided (which have all been documented in the lucene-cli docs).
    • The code is also embedded as resources inside of lucene-cli so it can be exported on disk by running the --output-source-code option
    • There are references in the documentation that link to the code files in the Lucene.Net.Demo project in the repository so the user has an alternate way to see the demo code than installing lucene-cli
    • Most of the demo code has also been added to Markdown files (https://github.com/apache/lucenenet/issues/284#issuecomment-806268971), but AFAIK these are not being referenced in other documentation

    Problems

    • The demos are not mentioned anywhere on the home page, you have to drill down pretty deep in the documentation to find them
    • The source code alone is not enough for the user to take advantage of it by using dotnet run or putting it in Visual Studio (a .csproj file is also required, I believe, but there may also be other dependencies to investigate)
    • The links to the demo source code are not functional yet because docs.ps1 build doesn't replace the tags in URLs
    • Keeping the demos in Markdown means that they could go out of date when we upgrade to the next version of Lucene. Note that Microsoft has info about how to include code from a .cs file in the docs dynamically without copy and paste.
    • The example code on the home page is missing details about what NuGet packages to reference, what namespaces to import, and a basic way to make the code run (#453)
    • There is also an external demo project, LuceneNetDemo that is being kept up to date, but is not being referenced in documentation. Instructions on how to get it running are also pretty thin.

    The bottom line is that it is not easy to locate the demos and if you do find them there is little information about how to run them.

    Research

    We need to research to find the best way to get these demos running on developer machines and update the documentation and website to make this experience go smoother. I suspect we can use the dotnet new command in combination with lucene-cli and dotnet run to do most of the heavy lifting. But it also might make sense to use the templating feature of dotnet new. We should investigate how other open source projects are providing runnable demos to end users to see if there are any ideas we can borrow.

    It would also certainly be useful to run the Java demos to determine exactly how they are intended to be used.

    For Lucene.Net.Benchmarks, we have similar issues because it is unclear how the app is intended to be used by end users. So, what is learned about the demos will also help us work out how to create the optimal user experience for Lucene.Net.Benchmarks.

    • Like Lucene.Net.Demo, we compile Lucene.Net.Benchmarks into lucene-cli.
    • Lucene.Net.Benchmarks is not exportable as source code.
    • Unlike Lucene.Net.Demo, Lucene.Net.Benchmarks is compiled and distributed on NuGet.

    It is clear that Lucene.Net.Benchmark must be referenced to extend types, but the extensions need to be loaded via Reflection on the command line to run (#305). It is not clear if there was a workflow in Java where you are expected to compile from source to do the benchmarks, although it may also have been possible for end users to make their own runner implementation.

    Project Infrastructure docs is:enhancement pri:high up-for-grabs 
    opened by NightOwl888 0
  •  When is the release version planned?

    When is the release version planned?

    Due to some annoying rules, we have to use release version. So, when is the release version planned?

    is:question 
    opened by VollyBird 2
  • Query/Filter Casting

    Query/Filter Casting

    Several APIs return type Query or another type such as DocIdSet with the expectation that the consumer will cast it to the appropriate type of query when utilizing it. This can be observed by analyzing the Lucene.Net.Tests.Queries project. Some known examples are:

    • MoreLikeThis.Like() (Returns Query)
    • Query.Rewrite() (Returns Query)
    • IndexSearcher.Rewrite() (Returns Query)
    • Filter.GetDocIdSet() (Returns DocIdSet)

    While there is nothing really broken with this, it is less than ideal. There is at least one way we can make these APIs better for the consumer: by providing generic overloads with a constraint.

    public T Like<T>(TextReader r, string fieldName) where T : Query
    {
        Return (T)Like(r, fieldName);
    }
    

    This improves things a bit by enforcing correct casting at compile time instead of getting casting exceptions at runtime.

    design is:enhancement pri:normal up-for-grabs 
    opened by NightOwl888 0
  • Remove unnecessary casting from Clone() methods

    Remove unnecessary casting from Clone() methods

    In Java, overridden methods in subclasses can return a different type than the base class. Therefore, it is common in Java to declare the clone() method with the subclass type so there is no need for the consumer to cast the return value of the method.

    In .NET, this only works if a class is sealed, since it would otherwise constrain the subclass to that of the base class type. While we don't use the ICloneable interface in .NET per Microsoft's recommendation, we left the return type as object for compatibility and provide an option for 3rd parties to create a custom compile that implements ICloneable in all of the appropriate places. While this affects usability somewhat by requiring a cast, the fact of the matter is object return type is the only thing we can do consistently across the API even if we took out the ICloneable option.

    Although the return type of Clone() is always object, many of the original casts to a specific type were carried over from Java and they can now be removed. In particular, there are some calls to MemberwiseClone() that are cast to a more specific type even though the return type is object and there is no need to set any of its members.

            public virtual object Clone()
            {
                return (MergeScheduler)base.MemberwiseClone();
            }
    
    design performance pri:normal 
    opened by NightOwl888 6
  • FieldDoc.Fields boxing issue

    FieldDoc.Fields boxing issue

    The Lucene.Net.Search.FieldDoc.Fields property returns object[]. However, according to the documentation, it is designed to be used with string, int, or float.

    In Java, this was not an issue because there are numeric reference types that primitive numbers are wrapped in when passed around. But, since it would not be a good user experience to require wrapper classes for numeric types in .NET we should solve the boxing issue another way.

    design is:enhancement performance pri:high 
    opened by NightOwl888 3
  • FunctionValues.ObjectVal() boxing issue

    FunctionValues.ObjectVal() boxing issue

    The Lucene.Net.Queries.Function.FunctionValues.ObjectVal() method returns object as its name implies. However, it often returns float or long?, which leads to expensive boxing.

    In Java, this was not an issue because there are numeric reference types that primitive numbers are wrapped in when passed around. But, since it would not be a good user experience to require wrapper classes for numeric types in .NET we should solve the boxing issue another way.

    Some Options

    1. Make the FunctionValues class generic (may not be possible due to how strict generics are in C#).
    2. Make the ObjectVal() method return a generic.
    3. Create a new method named GetValue<T>() (or I guess if we are following the lousy convention, GetVal<T>()) that returns a generic for use in .NET and recommend not to use the original one.
    design is:enhancement performance pri:high up-for-grabs 
    opened by NightOwl888 0
Releases(Lucene.Net_4_8_0_beta00014)
  • Lucene.Net_4_8_0_beta00014(Mar 28, 2021)

    This release contains bug fixes and minor performance improvements

    Known Issues

    • The lucene-cli tool requires an appsettings.json file, but none was shipped. Upon running lucene on the command line, the following error will be presented:

      F:\Projects\lucenenet>lucene
      Unhandled exception. System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional. The         physical path is 'C:\Users\shad\.dotnet\tools\.store\lucene-cli\4.8.0-beta00010\lucene-cli\4.8.0-beta00010\tools\netcoreapp3.1\any\appsettings.json'.
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
      at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
      at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
      at Lucene.Net.Cli.Program.Main(String[] args) in D:\a\1\s\src\dotnet\tools\lucene-cli\Program.cs:line 27
      

      Adding a text file named appsettings.json to the location specified in the error message with opening and closing brackets will prevent the exception.

      appsettings.json

      {
      }
      

      IMPORTANT: There must be at least opening and closing curly brackets in the file, or it won't be parsed as valid JSON.

    Benchmarks (from #310)

    Index Files

    Click to expand
    
    BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.867 (2004/?/20H1)
    Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
    .NET Core SDK=5.0.104
      [Host]          : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00005 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00006 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00007 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00008 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00009 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00010 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00011 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00012 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00013 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00014 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
    
    InvocationCount=1  IterationCount=15  LaunchCount=2  
    UnrollFactor=1  WarmupCount=10  
    
    

    | Method | Job | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |----------- |---------------- |---------:|----------:|----------:|-----------:|----------:|----------:|----------:| | IndexFiles | 4.8.0-beta00005 | 905.6 ms | 131.82 ms | 197.30 ms | 43000.0000 | 8000.0000 | 7000.0000 | 220.99 MB | | IndexFiles | 4.8.0-beta00006 | 707.1 ms | 18.57 ms | 26.04 ms | 44000.0000 | 8000.0000 | 7000.0000 | 220.99 MB | | IndexFiles | 4.8.0-beta00007 | 712.2 ms | 16.45 ms | 23.06 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.04 MB | | IndexFiles | 4.8.0-beta00008 | 785.7 ms | 17.37 ms | 25.46 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.54 MB | | IndexFiles | 4.8.0-beta00009 | 824.9 ms | 32.86 ms | 48.17 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.34 MB | | IndexFiles | 4.8.0-beta00010 | 789.6 ms | 16.40 ms | 24.04 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.35 MB | | IndexFiles | 4.8.0-beta00011 | 805.4 ms | 21.26 ms | 31.82 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.37 MB | | IndexFiles | 4.8.0-beta00012 | 827.8 ms | 13.95 ms | 20.89 ms | 56000.0000 | 7000.0000 | 6000.0000 | 287.03 MB | | IndexFiles | 4.8.0-beta00013 | 793.6 ms | 13.63 ms | 19.55 ms | 44000.0000 | 8000.0000 | 7000.0000 | 220.22 MB | | IndexFiles | 4.8.0-beta00014 | 812.0 ms | 21.97 ms | 30.79 ms | 44000.0000 | 8000.0000 | 7000.0000 | 220.29 MB |

    Search Files

    Click to expand
    
    BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.867 (2004/?/20H1)
    Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
    .NET Core SDK=5.0.104
      [Host]          : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00005 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00006 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00007 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00008 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00009 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00010 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00011 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00012 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00013 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
      4.8.0-beta00014 : .NET Core 3.1.13 (CoreCLR 4.700.21.11102, CoreFX 4.700.21.11602), X64 RyuJIT
    
    IterationCount=15  LaunchCount=2  WarmupCount=10  
    
    

    | Method | Job | Mean | Error | StdDev | Median | Gen 0 | Gen 1 | Gen 2 | Allocated | |------------ |---------------- |---------:|----------:|----------:|---------:|-----------:|----------:|------:|----------:| | SearchFiles | 4.8.0-beta00005 | 421.1 ms | 111.47 ms | 163.38 ms | 326.3 ms | 18000.0000 | 1000.0000 | - | 82.11 MB | | SearchFiles | 4.8.0-beta00006 | 349.8 ms | 24.03 ms | 35.97 ms | 338.9 ms | 18000.0000 | 1000.0000 | - | 82.11 MB | | SearchFiles | 4.8.0-beta00007 | 333.6 ms | 17.36 ms | 25.98 ms | 336.8 ms | 18000.0000 | 1000.0000 | - | 81.9 MB | | SearchFiles | 4.8.0-beta00008 | 191.7 ms | 7.17 ms | 10.51 ms | 187.9 ms | 17000.0000 | 1000.0000 | - | 80.13 MB | | SearchFiles | 4.8.0-beta00009 | 186.6 ms | 8.56 ms | 12.55 ms | 184.0 ms | 17000.0000 | 1000.0000 | - | 80.13 MB | | SearchFiles | 4.8.0-beta00010 | 182.2 ms | 6.69 ms | 9.16 ms | 181.6 ms | 17000.0000 | 1000.0000 | - | 79.85 MB | | SearchFiles | 4.8.0-beta00011 | 208.9 ms | 17.73 ms | 26.54 ms | 207.9 ms | 17000.0000 | 1000.0000 | - | 79.85 MB | | SearchFiles | 4.8.0-beta00012 | 192.3 ms | 10.99 ms | 16.46 ms | 187.8 ms | 18000.0000 | 1000.0000 | - | 81.11 MB | | SearchFiles | 4.8.0-beta00013 | 177.4 ms | 7.74 ms | 11.59 ms | 175.1 ms | 14000.0000 | 1000.0000 | - | 65.78 MB | | SearchFiles | 4.8.0-beta00014 | 172.7 ms | 5.93 ms | 8.88 ms | 168.9 ms | 14000.0000 | 1000.0000 | - | 65.78 MB |

    Change Log

    Breaking Changes

    • #424 - Deprecated TaskMergeScheduler, a merge scheduler that was added to support .NET Standard 1.x
    • #424 - Lucene.Net.TestFramework: Removed the public LuceneTestCase.ConcurrentMergeSchedulerFactories class

    Bugs

    • #405, #415 - Lucene.Net.Index.DocTermOrds: Fixed issue with enumerator (OrdWrappedTermsEnum) incorrectly returning true when the value is null.
    • #427 - Lucene.Net.Analysis.Common: Fixed TestRollingCharBuffer::Test() to prevent out of memory exceptions when running with Verbose enabled
    • #434, #418 - Hunspell affixes' file parsing corrupts some affixes' conditions
    • #434, #419 - HunspellStemFilter does not work with zero affix
    • #439 - Lucene.Net.Facet.Taxonomy.CachedOrdinalsReader: Fixed synchronization issue between adding new items to the cache and reading RamBytesUsed method
    • #439, #417, #319 - Lucene.Net.Spatial.Util.ShapeFieldCacheProvider: Fixed atomicity issue with loading the cache by using Lazy<T>.
    • #441 - Lucene.Net.TestFramework.Support.Confguration.TestConfigurationFactory: Use Lazy<T> to ensure the configurationCache.GetOrAdd() factory is atomic.
    • #441 - Lucene.Net.TestFramework.Search.ShardSearchingTestBase: Fixed possibleKeyNotFoundExceptionwhen getting the value fromcollectionStatisticsCache`
    • #441, #417, #319 - Lucene.Net.Spatial.Prefix.PrefixTreeFactory: Use Lazy<T> in ConcurrentDictionary to make the valueFactory atomic.
    • #443 - Lucene.Net.Benchmark.ByTask.Feeds.SpatialDocMaker: SinceDictionary<TKey, TValue>.this[key]is not marked virtual in .NET, subclassingDictionary<string, string>is not a valid approach. So we implementIDictionary<string, string>` instead.
    • #416 - CLI Documentation issue - environment variable token not replaced.
    • #450 - Lucene.Net.Facet - Reverted locking in to the state it was in Lucene 4.8.1, however we are still making use of ReaderWriterLockSlim to improve read performance of caches. Also, removed the 1 second lock timeout from Cl2oTaxonomyWriterCache.

    Improvements

    • #269 - Added [AwaitsFix] attribute to known failing tests
    • #391 - Improved plugins in DocFx when generating API docs
    • #392 - Enabled GitHub Actions to Run Tests on Pull Request
    • #395 - Improved performance of build pipeline by publishing the whole solution in one step instead of one project at a time
    • #395 - Fixed dependency NuGet package version conflicts
    • #395 - Added crash and hang detection to the test runs
    • #395 - Upgraded to the latest dotnet CLI commands dotnet build and dotnet test rather than dotnet msbuild and dotnet vstest
    • #411, #259 - Reviewed tests for Lucene.Net.Tests.Facet
    • #412, #406 - Upgraded NUnit to 3.13.1 and NUnit3TestAdapter to 3.17.0 to make Console.WriteLine() work in unit tests.
    • #414, #259 - Review of tests for Lucene.Net.Tests.Join
    • #420, #259 - Review of tests for Lucene.Net.Tests.Classification
    • #422 - Lucene.Net.Classification: Removed leading underscore from private/internal member variables
    • #423 - Reduced casting
    • #423 - azure-pipelines.yml: Added RunX86Tests option to explicitly enable x86 tests without having to run a full nightly build
    • #425, #259 - Review of tests for Lucene.Net.Tests.Codecs
    • #426 - Changed multiple naming conventions of anonymous classes to just use the suffix AnonymousClass
    • #426 - Changed accessibility of anonymous classes to private
    • #427, #259 - Review of tests for Lucene.Net.Tests.Queries
    • #433, #430 - Removed FEATURE_CLONEABLE and the MSBuild property IncludeICloneable
    • #435, #259 - Review of tests for Lucene.Net.Tests.Expressions
    • #438 - Don't insert extra newline in TFIDFSim's score explanation (this minor change had already been done to Lucene 5.0, so we are back-porting it to 4.8.0)
    • #439 - Lucene.Net.Util.VirtualMethod: Removed unnecessary call to Convert.ToInt32()
    • #439 - Lucene.Net.Util.AttributeSource: Restored comment from Lucene indicating it doesn't matter if multiple threads compete to populate the ConditionalWeakTable.
    • #440 - SWEEP: Reviewed catch blocks and made improvements to preserve stack details.
    • #441, #417 - Lucene.Net.Analysis.OpenNLP.Tools.OpenNLPOpsFactory: Simplified logic by using GetOrAdd() instead of TryGetValue.
    • #441 - Lucene.Net.TestFramework.Util (LuceneTestCase + TestUtil): Refactored the CleanupTemporaryFiles() method to be more in line with the original Java implementation, including not allowing new files/directories to be added to the queue concurrently with the deletion process.
    • #441 - PERFORMANCE: Lucene.Net.Join.ToParentBlockJoinCollector: Changed from ConcurrentQueue<T> to Queue<T> because we are dealing with a collection declared within the same method so there is no reason for the extra overhead.
    • #441 - PERFORMANCE: Lucene.Net.Tests.Suggest.Spell.TestSpellChecker: Replaced ConcurrentBag<T> with ConcurrentQueue because we need to be sure the underlying implementation guarantees order and the extra call to Reverse() was just slowing things down.
    • #441 - Lucene.Net.TestFramework.Search.ShardSearchingTestBase: Display the contents of the collection to the console using Collections.ToString().
    • #441 - Lucene.Net.Search.SearcherLifetimeManager: Added comment to indicate the reason we useLazy` is to make the create operation atomic.
    • #441 - Directory.Build.Targets: Added FEATURE_DICTIONARY_REMOVE_CONTINUEENUMERATION so we can support this feature in .NET 5.x + when we add a target.
    • #442 - PERFORMANCE: Lucene.Net.Search.Suggest.Fst.FSTCompletion: Use Stack<T> rather than List<T>.Reverse(). Also, removed unnecessary lock in CheckExistingAndReorder(), as it is only used in a single thread at a time.
    • #442 - PERFORMANCE: Lucene.Net.Search.Suggest.SortedInputEnumerator: Removed unnecessary call to Reverse() and allocation of HashSet<T>
    • #444, #272 - PERFORMANCE: Lucene.Net.Search.FieldCacheImpl: Reverted locking back to the state of Lucene 4.8.0.
    • #445 - Removed FEATURE_THREAD_INTERRUPT since all supported targets now support thread interrupts. Note also that Lucene depends on thread interrupts to function properly, so disabling this feature would be invalid.
    • #448 - DOCS: Added migration guide for users migrating from Lucene.NET 3.0.3 to Lucene.NET 4.8.0.
    • #396 - DOCS: Create branching scheme to track changes in docuentation between different Lucene versions and removed the JavaDocToMarkdownConverter tool from the normal build workflow of the API docs. This frees us up to update the "namespace" documentation with .NET-specific information and code examples.
    • Upgraded J2N NuGet package dependency to 2.0.0-beta-0012
    • Upgraded ICU4N NuGet package dependency to 60.1.0-alpha.254
    • Upgraded Morfologik.Stemming package dependency to 2.1.7-beta-0002

    New Features

    • #385, #362 - Lucene.Net.Documents.Document: Added culture-sensitive overloads of GetValues(), Get() and GetStringValue() that accept format and IFormatProvider and implemented IFormattable on Document and LazyDocument.
    • #404 - Added Commit() method to AnalyzingInfixSuggester (from LUCENE-5889)
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00013(Nov 16, 2020)

    This release contains important bug fixes and performance enhancements.

    Known Issues

    • The lucene-cli tool requires an appsettings.json file, but none was shipped. Upon running lucene on the command line, the following error will be presented:

      F:\Projects\lucenenet>lucene
      Unhandled exception. System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional. The         physical path is 'C:\Users\shad\.dotnet\tools\.store\lucene-cli\4.8.0-beta00010\lucene-cli\4.8.0-beta00010\tools\netcoreapp3.1\any\appsettings.json'.
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
      at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
      at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
      at Lucene.Net.Cli.Program.Main(String[] args) in D:\a\1\s\src\dotnet\tools\lucene-cli\Program.cs:line 27
      

      Adding a text file named appsettings.json to the location specified in the error message with opening and closing brackets will prevent the exception.

      appsettings.json

      {
      }
      

      IMPORTANT: There must be at least opening and closing curly brackets in the file, or it won't be parsed as valid JSON.

    • J2N versions prior to version 2.0.0-beta-0012 had an infinite recursion bug on Xamarin.Android which caused fatal crashes in Lucene.NET. Upgrading J2N to 2.0.0-beta-0012 or higher will prevent these crashes from occurring.

    Benchmarks (from #310)

    Index Files

    Click to expand
    
    BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.630 (2004/?/20H1)
    Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
    .NET Core SDK=5.0.100
      [Host]          : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00005 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00006 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00007 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00008 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00009 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00010 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00011 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00012 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00013 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
    
    InvocationCount=1  IterationCount=15  LaunchCount=2  
    UnrollFactor=1  WarmupCount=10  
    
    

    | Method | Job | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |----------- |---------------- |---------:|---------:|---------:|-----------:|----------:|----------:|----------:| | IndexFiles | 4.8.0-beta00005 | 628.1 ms | 8.41 ms | 12.05 ms | 43000.0000 | 8000.0000 | 7000.0000 | 220.82 MB | | IndexFiles | 4.8.0-beta00006 | 628.3 ms | 13.19 ms | 19.33 ms | 44000.0000 | 8000.0000 | 7000.0000 | 220.67 MB | | IndexFiles | 4.8.0-beta00007 | 617.2 ms | 8.44 ms | 11.83 ms | 44000.0000 | 8000.0000 | 7000.0000 | 220.73 MB | | IndexFiles | 4.8.0-beta00008 | 620.6 ms | 5.62 ms | 8.41 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.06 MB | | IndexFiles | 4.8.0-beta00009 | 632.8 ms | 12.57 ms | 18.43 ms | 44000.0000 | 8000.0000 | 7000.0000 | 220.95 MB | | IndexFiles | 4.8.0-beta00010 | 862.3 ms | 51.13 ms | 74.95 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.22 MB | | IndexFiles | 4.8.0-beta00011 | 636.5 ms | 11.06 ms | 15.87 ms | 44000.0000 | 8000.0000 | 7000.0000 | 221.09 MB | | IndexFiles | 4.8.0-beta00012 | 668.8 ms | 14.78 ms | 21.66 ms | 56000.0000 | 7000.0000 | 6000.0000 | 286.63 MB | | IndexFiles | 4.8.0-beta00013 | 626.7 ms | 7.78 ms | 10.91 ms | 43000.0000 | 8000.0000 | 7000.0000 | 219.8 MB |

    Search Files

    Click to expand
    
    BenchmarkDotNet=v0.12.1, OS=Windows 10.0.19041.630 (2004/?/20H1)
    Intel Core i7-8850H CPU 2.60GHz (Coffee Lake), 1 CPU, 12 logical and 6 physical cores
    .NET Core SDK=5.0.100
      [Host]          : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00005 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00006 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00007 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00008 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00009 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00010 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00011 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00012 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
      4.8.0-beta00013 : .NET Core 3.1.9 (CoreCLR 4.700.20.47201, CoreFX 4.700.20.47203), X64 RyuJIT
    
    IterationCount=15  LaunchCount=2  WarmupCount=10  
    
    

    | Method | Job | Mean | Error | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated | |------------ |---------------- |---------:|--------:|---------:|-----------:|----------:|------:|----------:| | SearchFiles | 4.8.0-beta00005 | 274.8 ms | 7.01 ms | 10.28 ms | 18000.0000 | 1000.0000 | - | 82.12 MB | | SearchFiles | 4.8.0-beta00006 | 283.4 ms | 7.78 ms | 11.64 ms | 18000.0000 | 1000.0000 | - | 82.13 MB | | SearchFiles | 4.8.0-beta00007 | 291.5 ms | 8.91 ms | 13.33 ms | 18000.0000 | 1000.0000 | - | 81.9 MB | | SearchFiles | 4.8.0-beta00008 | 162.3 ms | 5.50 ms | 8.23 ms | 17000.0000 | 1000.0000 | - | 80.13 MB | | SearchFiles | 4.8.0-beta00009 | 165.6 ms | 2.61 ms | 3.90 ms | 17000.0000 | - | - | 80.13 MB | | SearchFiles | 4.8.0-beta00010 | 159.4 ms | 2.84 ms | 4.17 ms | 17000.0000 | 1000.0000 | - | 79.85 MB | | SearchFiles | 4.8.0-beta00011 | 160.8 ms | 1.93 ms | 2.77 ms | 17000.0000 | 1000.0000 | - | 79.85 MB | | SearchFiles | 4.8.0-beta00012 | 169.2 ms | 6.48 ms | 9.49 ms | 18000.0000 | 1000.0000 | - | 81.11 MB | | SearchFiles | 4.8.0-beta00013 | 161.6 ms | 3.28 ms | 4.80 ms | 14000.0000 | 1000.0000 | - | 65.78 MB |

    Change Log

    Breaking Changes

    • Lucene.Net.Search.FieldCache: Added interface ICreationPlaceholder and changed CreationPlaceholder class to CreationPlaceHolder<TValue>.

    Bugs

    • #356 - Lucene.Net.Store.NativeFSLockFactory: Modified options to allow read access on non-Windows operating systems. This caused the copy constructor of RAMDirectory to throw "The process cannot access the file 'file path' because it is being used by another process" excpetions.
    • #296 - Lucene.Net.Util.Automaton.State: Removed Equals() implementation; it was intended to use reference equality as a unique key. This caused random IndexOperationExceptions to occur when using FuzzyTermsEnum/FuzzyQuery.
    • #387 - Fixed formatting in ArgumentException message for all analyzer factories so it will display the dictionary contents
    • #387 - Lucene.Net.Util.ExceptionExtensions.GetSuppressedAsList(): Use J2N.Collections.Generic.List<T> so the call to ToString() will automatically list the exception messages
    • #387 - Lucene.Net.TestFramework.Analysis.MockTokenizer: Pass the AttributeFactory argument that is provided as per the documentation comment. Note this bug exists in Lucene 4.8.0, also.
    • #387 - Lucene.Net.Analysis.Common.Tartarus.Snowball.Among: Fixed MethodObject property to return private field instead of itself
    • #387 - Lucene.Net.Document.CompressionTools: Pass the offset and length to the underlying MemoryStream
    • #388 - Downgraded minimum required Microsoft.Extensions.Configuration version to 2.0.0 on .NET Standard 2.0 and 2.1

    Improvements

    • Updated code examples on website home page
      1. Show cross-OS examples of building Directory paths
      2. Demonstrate where to put using statements
      3. Removed LinqPad's Dump() method and replaced with Console.WriteLine() for clarity
      4. Fixed syntax error in initialization example of MultiPhraseQuery
    • Upgraded NuGet dependency J2N to 2.0.0-beta-0010
    • Upgraded NuGet dependency ICU4N to 60.1.0-alpha.353
    • Upgraded NuGet dependency Morfologik.Stemming to 2.1.7-beta-0001
    • #344 - PERFORMANCE: Lucene.Net.Search.FieldCacheImpl: Removed unnecessary dictionary lookup
    • #352 - Added Azure DevOps tests for x86 on all platforms
    • #348 - PERFORMANCE: Reduced FieldCacheImpl casting/boxing
    • #355 - Setup nightly build (https://dev.azure.com/lucene-net/Lucene.NET/_build?definitionId=4)
    • PERFORMANCE: Lucene.Net.Util.Automaton.SortedInt32Set: Removed unnecessary IEquatable<T> implementations and converted FrozenInt32Set into a struct.
    • PERFORMANCE: Lucene.Net.Util.Bits: Removed unnecessary GetHashCode() method from MatchAllBits and MatchNoBits (didn't exist in Lucene)
    • Lucene.Net.Util.Counter: Changed Get() to Value property and added implicit operator.
    • #361 - Make CreateDirectory() method virtual so that derived classes can provide their own Directory implementation, allowing for benchmarking of custom Directory providers (e.q LiteDB)
    • #346, #383 - PERFORMANCE: Change delegate overloads of Debugging.Assert() to use generic parameters and string.Format() to reduce allocations. Use J2N.Text.StringFormatter to automatically format arrays and collections so the processing of converting it to a string is deferred until an assert fails.
    • #296 - PERFORMANCE: Lucene.Net..Index: Calling IndexOptions.CompareTo() causes boxing. Added new IndexOptionsComparer class to be used in codecs instead.
    • #387 - Fixed or Suppressed Code Analysis Rules
      • CA1012: Abstract types should not have constructors
      • CA1052: Static holder types should be Static or NotInheritable
      • CA1063: Implement IDisposable Properly (except for IndexWriter). Partially addresses #265.
      • CA1507: Use nameof instead of string (#366)
      • CA1802: Use Literals Where Appropriate
      • CA1810: Initialize reference type static fields inline
      • CA1815: Override equals and operator equals on value types
      • CA1819: Properties should not return arrays
      • CA1820: Test for empty strings using string length
      • CA1822: Mark members as static
      • CA1825: Avoid zero-length array allocations
      • CA2213: Disposable fields should be disposed (except for IndexWriter and subclasses which need more work)
      • IDE0016: use throw expression (#368)
      • IDE0018: Inline variable declaration
      • IDE0019: Use pattern matching to avoid 'is' check followed by a cast
      • IDE0020: Use pattern matching to avoid 'is' check followed by a cast
      • IDE0021: Use block body for constructors
      • IDE0025: Use expression body for properties
      • IDE0027: Use expression body for accessors
      • IDE0028: Use collection initializers
      • IDE0029: Use coalesce expression
      • IDE0030: Use coalesce expression (nullable)
      • IDE0031: Use null propagation
      • IDE0034: Simplify 'default' expression
      • IDE0038: Use pattern matching to avoid 'is' check followed by a cast
      • IDE0039: Use local function
      • IDE0040: Add accessibility modifiers
      • IDE0041: Use is null check
      • IDE0049: Use language keywords instead of framework type names for type references
      • IDE0051: Remove unused private member
      • IDE0052: Remove unread private member
      • IDE0059: Remove unnecessary value assignment
      • IDE0060: Remove unused parameter
      • IDE0063: Use simple 'using' statement
      • IDE0071: Simplify interpolation
      • IDE1005: Use conditional delegate call
      • IDE1006: Naming Styles
    • #387 - Removed dead code/commented code
    • #387 - PERFORMANCE: Added aggressive inlining in Codecs and Util namespaces
    • #387 - Simplified reuse logic of TermsEnum subclasses
    • #387 - PERFORMANCE: Lucene.Net.Index.DocValuesProducer: Optimized checks in AddXXXField() methods
    • #387 - PERFORMANCE: Lucene.Net.Index: Changed FieldInfos, FreqProxTermsWriterPerField, IndexWriter, LogMergePolicy, SegmentCoreReaders, and SegmentReader to take advantage of the fact that TryGetValue() returns a boolean
    • #370, #389 - Reverted FieldCacheImpl delegate capture introduced in #348
    • #390 - Added tests for .NET 5
    • #390 - Upgraded to C# LangVersion 9.0

    New Features

    • #358 - Added Community Links page to website
    • #359 - Added builds mailing list to website
    • #365 - Added "Fork me on GitHub" to website and API docs
    • Lucene.Net.TestFramework: Added Assert.DoesNotThrow() overloads
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00012(Sep 19, 2020)

    This release contains important bug fixes and performance enhancements.

    Known Issues

    • After installation, when upgrading from versions of Lucene.Net 4.8.0-beta00009 or higher may require a restart of all instances of Visual Studio in order to reload the code analysis analyzer.

    • The lucene-cli tool requires an appsettings.json file, but none was shipped. Upon running lucene on the command line, the following error will be presented:

      F:\Projects\lucenenet>lucene
      Unhandled exception. System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional. The         physical path is 'C:\Users\shad\.dotnet\tools\.store\lucene-cli\4.8.0-beta00010\lucene-cli\4.8.0-beta00010\tools\netcoreapp3.1\any\appsettings.json'.
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
      at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
      at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
      at Lucene.Net.Cli.Program.Main(String[] args) in D:\a\1\s\src\dotnet\tools\lucene-cli\Program.cs:line 27
      

      Adding a text file named appsettings.json to the location specified in the error message with opening and closing brackets will prevent the exception.

      appsettings.json

      {
      }
      

      IMPORTANT: There must be at least opening and closing curly brackets in the file, or it won't be parsed as valid JSON.

    • J2N versions prior to version 2.0.0-beta-0012 had an infinite recursion bug on Xamarin.Android which caused fatal crashes in Lucene.NET. Upgrading J2N to 2.0.0-beta-0012 or higher will prevent these crashes from occurring.

    Change Log

    Breaking Changes

    • Lucene.Net.Facet: Renamed LRUHashMap > LruDictionary. Changed all members to be virtual to allow users to provide their own LRU cache.
    • Lucene.Net.Facet.FacetsConfig: Removed ProcessSSDVFacetFields from public API (as was done in Lucene), avoid lock (this)
    • Lucene.Net.Facet.TaxonomyReader: Changed DoClose() to Dispose(bool) and implemented proper dispose pattern. Avoid lock (this).
    • Lucene.Net.Facet.WriterCache: Renamed NameInt32CacheLRU > NameIntCacheLru, NameHashInt32CacheLRU > NameHashInt32CacheLru. Refactored to utilize a generic type internally using composition to avoid boxing/unboxing without exposing the generic closing type publicly. Added public INameInt32CacheLru as a common interface between NameIntCacheLru and NameHashInt32CacheLru.
    • Lucene.Net.Facet.Taxonomy.TaxonomyReader: Restructured ChildrenIterator into ChildrenEnumerator
    • Lucene.Net.Facet.Taxonomy.CategoryPath: Changed FullPathLength from a method to a property
    • Lucene.Net.Facet.DrillSideways: Changed ScoreSubDocsAtOnce from a method to a property
    • Lucene.Net.Facet: Refactored OrdAndValue into a generic struct that can be used in both TopOrdAndSingleQueue and TopOrdAndInt32Queue. Added Insert method to Util.PriorityQueue<T> to allow adding value types without reading the previous value for reuse.
    • Lucene.Net.Analysis.Common.Miscellaneous.CapitalizationFilter: Changed default behavior to use invariant culture instead of the current thread's culture to match Lucene, which seems more natural when using filters inside of analyzers. This also fits more in line with how other filters are selected.
    • #279 - Lucene.Net.Analysis.Compound.Hyphenation.TernaryTree: Renamed Iterator > Enumerator, Keys() > GetEnumerator()
    • #279 - Lucene.Net.Benchmarks.ByTask.Feeds.DirContentSource: Renamed Iterator > Enumerator

    Bugs

    • #269 - Removed cast from NGramTokenAnymousInnerClassHelper::IsTokenChar(int) that was causing surrogate pairs to fail in the TestUTF8FullRange() tests of NGramTokenizerTest and EdgeNGramTokenizerTest
    • Fixed potential issue with ArgumentExceptions being thrown from char.ConvertToUtf32(string, int) by reverting back to CodePointAt() method in TestCharTokenizers.TestCrossPlaneNomalization().
    • Lucene.Net.QueryParser.Surround.Query.ComposedQuery::MakeLuceneSubQueriesField(): Added missing using block on enumerator
    • #296 - Fixed surrogate pair and culture-sensitivity issues with many analyzers.
    • Lucene.Net.Analysis.Common: Fixed classes that were originally using invariant culture to do so again. J2N's Character class default is to use the current culture, which had changed from the prior Character class from Lucene.Net.Support that used invariant culture. Fixes TestICUFoldingFilter::TestRandomStrings().
    • Lucene.Net.ICU: Fixed ThaiWordBreaker to account for surrogate pairs. Also added locking to help with thread safety. Note that the class is still not completely thread-safe, but this patch fixes the behavior.
    • Lucene.Net.Spatial.Util.ShapeFieldCache: Removed unnecessary array allocation
    • Lucene.Net.TestFramework: Fixed LineFileDocs to read byte by byte the same way that Lucene does, except using a BufferedStream to improve performance.
    • Lucene.Net.TestFramework: Fixed NightlyAttribute, WeeklyAttribute, AwaitsFixAttribute, and SlowAttribute so they work at the class level
    • Lucene.Net.Analysis.Icu.Segmentation.ICUTokenizer: Corrected call to ICU4N.UChar.IsWhiteSpace() rather than System.Char.IsWhiteSpace(), which may return different results.
    • Lucene.Net.TestFramework.Search.SearchEquivalenceTestBase: Fixed exception when using OpenBitSet.FastGet() instead of OpenBitSet.Get(), since the size of the bit set is unknown.
    • Lucene.Net.Index.DocumentsWriterFlushControl: Fixed issue due to misbehaving locking on Monitor.TryEnter(), the code was restructured to disallow any thread that doesn't have a lock into InternalTryCheckoutForFlush() so the threads do note compete for a lock.
    • #274 - Lucene.Net.Facet: Fixed null reference exception in DrillSidewaysScorer from patch in Lucene 4.10.4 https://issues.apache.org/jira/browse/LUCENE-6001
    • Lucene.Net.Facet.Taxonomy.WriterCache.Cl2oTaxonomyWriterCache: Fixed locking on Dispose() method and made it safe to call dispose multiple times
    • Reviewed and added asserts that existed in Lucene and were missing in Lucene.NET. Effectively, this meant we were missing several test conditions that have now been put into place.
    • Lucene.Net.ICU: Added locking to ICUTokenizer to only allow a single thread to manipulate the BreakIterator at a time. This is a temporary fix to get the tests to pass until a solution is found for making BreakIterator threadsafe.
    • #332 - Lucene.Net.Replicator: Fixed an issue in IndexInputStream that meant the read method could return a number larger than what was passed in for read count or what the buffer could hold, it should instead return the total number of bytes that was read into the buffer, which logically can't be bigger than the buffer it self.
    • Lucene.Net.Tests.Index.TestIndexWithThreads::TestRollbackAndCommitWithThreads(): Must catch and ignore AssertionException, as was done in Lucene
    • Lucene.Net.Search.TopScoreDocCollector: Disabled optimizations on .NET Framework because of float comparison failures on x86 in Release mode. Fixes TestSearchAfter::TestQueries(), TestTopDocsMerge::TestSort_1(), TestTopDocsMerge::TestSort_2().
    • Lucene.Net.Sandbox.Queries.SlowFuzzyTermsEnum: Disabled optimizations on .NET Framework because of float comparison failures on x86 in Release mode. Fixes TestTokenLengthOpt().
    • Lucene.Net.Search.FuzzyTermsEnum: Disabled optimizations for Accept() method on .NET Framework because of float comparison failures on x86 in Release mode. Fixes TestTokenLengthOpt().
    • Fixed several references to J2N.BitConversion that were calling the overload that normalizes NaN when they should have been calling the raw bit conversion instead (as was done in Lucene).
    • #323 - Lucene.Net.Configuration: Removed the IConfigurationRoot interface from the ConfigurationRoot class when targeting a version of Microsoft.Extensions.Configuration less than 2.0. This will allow the end user to upgrade Microsoft.Extensions.Configuration seamlessly to versions 2.0 or higher.
    • #286 - Lucene.Net.CodeAnalysis: Separated CSharp and VisualBasic into different assemblies to prevent cross-language dependency issues when using analyzers

    Improvements

    • #261 PERFORMANCE - Fixed FSTTester to delete while iterating forward instead of using .ElementAt() to iterate in reverse, which takes about 3x longer
    • #261 PERFORMANCE - Lucene.Net.Facet.Taxonomy.WriterCache.NameInt32CacheLRU: Changed from Dictionary to ConcurrentDictionary so we can delete items from the cache while forward iterating through it.
    • #261 PERFORMANCE - Lucene.Net.Index.FieldInfos: Changed Builder.FieldInfo() method to TryGetFieldInfo() to optimize check for value
    • Upgraded NuGet dependency J2N to 2.0.0-beta-0009
    • Upgraded NuGet dependency ICU4N to 60.1.0-alpha.352
    • Upgraded NuGet dependency Morfologik.Stemming to 2.1.6-beta-0007
    • #261 PERFORMANCE - Use J2N's ICollection<T>.ToArray() extension method that uses ICollection<T>.CopyTo(), which takes precedence over the LINQ IEnumerable<T>.ToArray() extension method. Benchmarks show about a 1/3 increase in performance.
    • #261 PERFORMANCE - Lucene.Net.Support.IO.FileSupport::CreateTempFile(): Optimized the check for invalid characters
    • Directory.Build.props: Disabled warnings for features that require .NET Standard 2.1
    • #261 PERFORMANCE - Eliminated several calls to FirstOrDefault(), LastOrDefault(), Skip(), First(), and Last()
    • Lucene.Net.Support.ListExtensions: Factored out BinarySearch in favor of implementation from J2N
    • Lucene.Net.Suggest.FreeTextSuggester: Converted from SubList().Clear() to RemoveRange()
    • #261 PERFORMANCE - Changed handling of LineFileDocs to unzip the file to a temp directory once per test run instead of using a MemoryStream to pick a random line from the file on each test. This significantly improves performance of many of the tests.
    • Lucene.Net.Analysis.Icu.Segmentation.ScriptIterator: Removed static constructor and initialized static state inline
    • Converted all explicit Analyzer classes to using the Analyzer.NewAnonymous() method to declare Analyzers inline.
    • #261 PERFORMANCE - Lucene.Net.Tests.Util.TestCollectionUtil: Optimized by using array instead of list for sorting tests
    • #261 PERFORMANCE - Lucene.Net.Util: Switched implementation of DisposableThreadLocal with that from RavenDB, with permission from its maintainers (https://issues.apache.org/jira/browse/LUCENENET-640?focusedCommentId=17033146&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17033146). The new implementation improves GC during several operations.
    • Lucene.Net.TestFramework.Util.LuceneTestCase: Removed TaskMergeScheduler completely from random testing
    • #261 PERFORMANCE - Moved scratch BytesRef instances outside of the loops that they were nested in so they can be reused in each iteration (as was done in Lucene)
    • #261 PERFORMANCE - Lucene.Net.Facet.Taxonomy.CachedOrdinalsReader: Refactored locking to make reads more efficient
    • #261 PERFORMANCE - Lucene.Net.Facet.Taxonomy.Directory.DirectoryTaxonomyReader: Refactored to use ReaderWriterLockSlim to make reads more efficient
    • #261 PERFORMANCE - Lucene.Net.Facet.Taxonomy.Directory.DirectoryTaxonomyWriter: Refactored locking for better efficiency
    • #265 - Lucene.Net.Facet.Taxonomy.WriterCache.Cl2oTaxonomyWriterCache: Added proper dispose pattern
    • #265 - Lucene.Net.Facet.Taxonomy.WriterCache.LruTaxonomyWriterCache: Added proper dispose pattern
    • Lucene.Net.Facet.Taxonomy.Directory.TaxonomyIndexArrays: Changed to use LazyInitializer and avoid lock (this)
    • #261 PERFORMANCE - Lucene.Net.Tests.Facet: Convert int to string in the invariant culture
    • Lucene.Net.Analysis.ICU: Updated Segmentation files to Lucene 8.6.1 to account for the latest features of ICU
    • #261 PERFORMANCE - Lucene.Net.Util.AttributeSource: Eliminated unnecessary try catch and made more efficient by using TryGetValue instead of ContainsKey followed by a lookup
    • Lucene.Net.Util: Streamlined DefaultAttributeFactory to make the get/update process of creating an attribute WeakReference atomic
    • #208 - Switch to simpler LIFO thread to ThreadState allocator during indexing. Technically, this is something from releases/lucene-solr/4.8.1, but profiling indicates it makes a huge difference in multithreaded scenarios
    • SWEEP - Removed unnecessary .NET Framework references from all test projects
    • Converted remaining compilation constants from target platforms to features to make it simpler to change targets. Eliminated references to NETSTANDARD.
    • Inverted logic so FEATURE_STACKTRACE is enabled rather than disabled when the System.Diagnositcs.StackTrace class is available.
    • Lucene.Net.Constants: Refactored to use System.Runtime.InteropServices.RuntimeInformation on .NET Framework
    • Lucene.Net.Expressions: Eliminated .NET settings file and reused JavascriptCompiler.properties file in .NET Framework so we don't have to branch for different target platforms. Simplified reading the settings by using J2N PropertyExtensions.
    • #261 PERFORMANCE - Lucene.Net.Support.AssemblyUtils: restructured to use IEnumerable<T> for deferred execution
    • #279 - Lucene.Net.Index.Terms/TermsEnum, Lucene.Net.Suggest: Refactored iterators into enumerators. Deprecated the iterators.
    • #279 - Lucene.Net.Util.FilterIterator<T>: Converted to FilterEnumerator<T> using a predicate passed into the constructor rather than having to subclass. Deprecated FilterIterator<T>. Swapped only usage in FieldFilterAtomicReader with a LINQ query/yield return, since performance is better.
    • #279 - Lucene.Net.Util.MergedIterator<T>: Converted to MergedEnumerator<T> and deprecated MergedIterator<T>
    • #279 - Lucene.Net.Codecs.Memory.DirectDocValuesConsumer: Renamed IteratorAnonymousInnerClassHelper > Enumerator, IterableAnonymousInnerClassHelper > EnumerableAnonymousInnerClassHelper

    New Features

    • Added DeadlockAttribute to identify tests that are known to have threading contention issues and may deadlock during test runs
    • Added ability to turn asserts on in a Release build by using the system property "assert": "true". This is necessary to ensure all of the test conditions are being hit in all builds and to enable more thorough CheckIndex from the command line.
    • Lucene.Net.TestFramework: Added ability to turn off asserts when running tests by ignoring a few tests that require the asserts to be enabled in order to pass. This makes it possible to ensure that Lucene.NET works properly with asserts disabled. This feature didn't exist in Lucene.
    • Lucene.Net.Search.FieldCacheDocIdSet: Added public constructor with predicate parameter for filtering without having to create a subclass
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00011(Jul 24, 2020)

    This release contains a critical patch for .NET Framework users that use Microsoft.Extensions.Configuration higher than version 1.1.2. See #311 for details.

    This release contains impactful performance enhancements.

    Known Issues

    • The lucene-cli tool requires an appsettings.json file, but none was shipped. Upon running lucene on the command line, the following error will be presented:

      F:\Projects\lucenenet>lucene
      Unhandled exception. System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional. The         physical path is 'C:\Users\shad\.dotnet\tools\.store\lucene-cli\4.8.0-beta00010\lucene-cli\4.8.0-beta00010\tools\netcoreapp3.1\any\appsettings.json'.
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
      at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
      at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
      at Lucene.Net.Cli.Program.Main(String[] args) in D:\a\1\s\src\dotnet\tools\lucene-cli\Program.cs:line 27
      

      Adding a text file named appsettings.json to the location specified in the error message with opening and closing brackets will prevent the exception.

      appsettings.json

      {
      }
      

      IMPORTANT: There must be at least opening and closing curly brackets in the file, or it won't be parsed as valid JSON.

    • J2N versions prior to version 2.0.0-beta-0012 had an infinite recursion bug on Xamarin.Android which caused fatal crashes in Lucene.NET. Upgrading J2N to 2.0.0-beta-0012 or higher will prevent these crashes from occurring.

    Change Log

    Breaking Changes

    • PERFORMANCE: Lucene.Net.Analysis.Compound: Changed protected m_tokens field from LinkedList<T> to Queue<T> for better throughput

    Bugs

    • #311 - Lucene.Net.Configuration: Removed IConfigurationBuilder implementation that prevents .NET Framework users from being able to upgrade to a higher version than 1.1.2.

    Improvements

    • PERFORMANCE: Lucene.Net.TestFramework: Reverted BaseTermVectorsFormatTestCase to use the original 5000 iterations instead of 500. Reverted TestUtil.RandomSimpleString(Random) to default to a maximum string length of 10 instead of 20, which was slowing down several tests.
    • PERFORMANCE: Lucene.Net.TestFramework: Refactored Assert class to use custom comparisons for all members, since NUnit's Assert implementation uses very slow fluent expressions to do comparisons, which are not practical to use inside of tight loops.
    • PERFORMANCE: Replaced LinkedList<T> withQueue` where there is a performance advantage
    • PERFORMANCE: Reduced memory allocations of CaseInsensitiveComparer by using its singleton static property
    • PERFORMANCE: Lucene.Net.Util.RamUsageEstimator: Switched back to System.Collections.Generic.Dictionary because indexer of J2N's dictionary is slower
    • PERFORMANCE: Lucene.Net.TestFramework.Util.Fst.FSTTester: Use System.Collections.Generic.Dictionary for better performance
    • PERFORMANCE: Switched all remaining tests from using NUnit.Framework.Assert to Lucene.Net.TestFramework.Assert, which are several orders of magnitude faster
    • PERFORMANCE: Lucene.Net.Facet: Optimized DirectoryTaxonomyReader by reducing locking, removing unnecessary casts, and using LazyInitializer for the taxonomy array initialization
    • PERFORMANCE: Lucene.Net.Tests.Analysis.Common: Changed Hunspell StemmerTestBase to use more optimized assert to compare arrays
    • SWEEP: Consolidated empty array creation code so it is more DRY
    • Upgraded to C# LangVersion 8.0
    • Lucene.Net.Tests.Support.TestApiConsistency: Added regex filter to exclude public fields from the scan
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00010(Jul 19, 2020)

    This release contains impactful performance enhancements.

    Known Issues

    • The lucene-cli tool requires an appsettings.json file, but none was shipped. Upon running lucene on the command line, the following error will be presented:

      F:\Projects\lucenenet>lucene
      Unhandled exception. System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional. The         physical path is 'C:\Users\shad\.dotnet\tools\.store\lucene-cli\4.8.0-beta00010\lucene-cli\4.8.0-beta00010\tools\netcoreapp3.1\any\appsettings.json'.
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
      at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
      at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
      at Lucene.Net.Cli.Program.Main(String[] args) in D:\a\1\s\src\dotnet\tools\lucene-cli\Program.cs:line 27
      

      Adding a text file named appsettings.json to the location specified in the error message with opening and closing brackets will prevent the exception.

      appsettings.json

      {
      }
      

      IMPORTANT: There must be at least opening and closing curly brackets in the file, or it won't be parsed as valid JSON.

    • J2N versions prior to version 2.0.0-beta-0012 had an infinite recursion bug on Xamarin.Android which caused fatal crashes in Lucene.NET. Upgrading J2N to 2.0.0-beta-0012 or higher will prevent these crashes from occurring.

    Change Log

    Breaking Changes

    • PERFORMANCE: Lucene.Net.Suggest: Narrowed return type of Contexts property from IEnumerable<BytesRef> to ICollection<BytesRef> to improve performance for certain operations
    • Lucene.Net.TestFramework.Util.LuceneTestCase: Refactored to correctly load static fields and converted them into static properties. The names were changed to match the conventions of .NET properties.
    • #261 PERFORMANCE: Changed to use BitSet instead of BitArray so we don't have to resort to slow extension methods for certain operations

    Bugs

    • Lucene.Net.Support.Collections::ToString(): Fixed overloads to write "null" when the collection passed is null rather than throw an exception
    • #267 - Lucene.Net.Codecs: Fixed testing condition for BaseTermVectorsFormatTestCase on TermVectorsReaders by throwing InvalidOperationException
    • #301 - After upgrading NuGet package dependencies for NUnit to 3.12.0, NUnit3TestAdapter to 3.16.1, and Microsoft.NET.Test.Sdk to 16.6.1, several false positives were noted due to invalid try-catch logic in tests

    Improvements

    • Added missing documentation table of contents links for lucene-cli, Lucene.Net.Demo, Lucene.Net.Queries, Lucene.Net.QueryParser, Lucene.Net.Replicator, Lucene.Net.Sandbox, Lucene.Net.Spatial, and Lucene.Net.Suggest.
    • #261 PERFORMANCE: - Removed all calls to Type.GetTypeInfo() and call properties/methods on the Type instance directly.
    • #261 PERFORMANCE: - Lucene.Net.Util.AttributeSource: Create built-in attributes directly rather than using Activator.CreateInstance()
    • #261 PERFORMANCE: - Lucene.Net.Util.AttributeSource: Optimized creation of string to identify attribute type based on attribute interface name
    • #295 PERFORMANCE: - Lucene.Net.TestFramework: Added overloads of Assert.AreEqual for collections where J2N's aggressive mode can be switched off
    • #295 PERFORMANCE: - Lucene.Net.TestFramework: Replaced overloads from NUnit.Framework.CollectionAssert with optimized implementations using J2N comparers
    • #295 PERFORMANCE: - Lucene.Net.TestFramework: Compile expensive string concatenation out of the release build by using System.Diagnostics.Debug.Assert
    • #295 PERFORMANCE: - Lucene.Net.Util.Automaton: Fixed State class to initialize and trim more efficiently. Fixes the performance of Lucene.Net.Util.Automaton.TestBasicOperations::TestEmptyLanguageConcatenate().
    • #295, #261: PERFORMANCE: - Reduced zero length array allocations
    • #295, #261: PERFORMANCE: - Reduced zero length collection allocations
    • #295, #261: PERFORMANCE: - Lucene.Net.TestFramework.Search.RandomSimilarityProvider::ToString(): Use StringBuilder for better efficiency
    • #295, #261: PERFORMANCE: - Lucene.Net.TestFramework.Util.LuceneTestCase: Cache codecType and similarityName as strings so they don't have to be regenerated for each test
    • #295, #261: PERFORMANCE: - Lucene.Net.TestFramework: Use Assert.IsFalse() rather than Assert.That()
    • #295, #261: PERFORMANCE: - Reduced calls to LINQ methods and expressions.
    • #295, #261: PERFORMANCE: - Lucene.Net.Index.Term: Optimized equality checking
    • #295, #261: PERFORMANCE: - Lucene.Net.Util (BytesRef + CharsRef): Implemented IEquatable<T>
    • #295, #261: PERFORMANCE: - Lucene.Net.Tests.Index.TestDocumentsWriterDeleteQueue: Updated comparisons to reduce memory allocations
    • #295, #261: PERFORMANCE: - Lucene.Net.TestFramework: Changed ConcurrentMergeSchedulerFactories.Values to only return the TaskMergeScheduler rarely, since it is no longer a default setting and is slowing down tests
    • #295, #261: PERFORMANCE: - Added some aggressive inlining
    • Lucene.Net.TestFramework: Added overloads of Assert.Throws to supply messages
    • Lucene.Net.TestFramework: Use the Microsoft.Extensions.Configuration.EnvironmentVariables provider instead of our custom one.
    • Lucene.Net.Analysis.Tokenizer: Allow enabling "asserts" for testing
    • azure-pipelines.yml: Changed to use pipeline caching instead of build caching for better performance
    • Lucene.Net.Tests (core) - Rearranged how test projects are split so parallel jobs can process them faster
    • Fixed up leading whitespace to always use spaces instead of tabs
    • Updated NuGet package dependency for J2N to 2.0.0-beta-0008
    • Lucene.Net.Tests.Analysis.Phonetic.Language.Bm.PhoneticEnginePeformanceTest::Test(): Changed to use Stopwatch for more accurate timing
    • Lucene.Net.Tests: Changed private/internal member variables to consistently use camelCase instead of PascalCase
    • SWEEP: Changed all properties to use expression style syntax, reordered to put get before set, and changed all backing field names back to their original without the "_Renamed" suffix
    • SWEEP: Removed fully-qualified exceptions and added using directives instead
    • PERFORMANCE: Pre-compile and statically cache regular expressions
    • PERFORMANCE: Lucene.Net.Analysis: Removed unnecessary allocations caused by calling ToString() rather than passing the ICharTermAttribute directly
    • PERFORMANCE: Lucene.Net.Analysis.Common: Slight optimization of ToUpper and ToLower methods, which are used in several analyzers
    • SWEEP: Removed .NET Standard 1.x/.NET Core 1.x support from all project files
    • SWEEP: Removed unused dependencies for .NET Framework
    • PERFORMANCE: Lucene.Net.Codecs.SimpleText: Using decimal is 30% faster than using BigInteger for addition and subtraction
    • PERFORMANCE: Applied Slow and Nightly attributes where applicable for better testing performance
    • Lucene.Net.TestFramework: Added logging of system properties to the output of tests
    • PERFORMANCE: Lucene.Net.Util.ArrayUtil::GetNaturalComparer<T>(): Use statically cached singleton instance of comparer
    • Upgraded NuGet dependency ICU4N to 60.1.0-alpha.351

    New Features

    • Lucene.Net.TestFramework.RandomExtensions: Added missing overload of NextInt64(long) to choose only max upper bound
    • Added a global Lucene.Net.Diagnostics.Debugging.AssertsEnabled static property that can be used to toggle "asserts" on and off in the release build, similar to how it works in Java. The setting can be injected by end users with the "assert" system property (which is a boolean).
    • Lucene.Net.TestFramework: Completed implementation of Nightly, Weekly, AwaitsFix and Slow attributes
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00009(Jun 30, 2020)

    Known Issues

    • The lucene-cli tool requires an appsettings.json file, but none was shipped. Upon running lucene on the command line, the following error will be presented:

      F:\Projects\lucenenet>lucene
      Unhandled exception. System.IO.FileNotFoundException: The configuration file 'appsettings.json' was not found and is not optional. The         physical path is 'C:\Users\shad\.dotnet\tools\.store\lucene-cli\4.8.0-beta00010\lucene-cli\4.8.0-beta00010\tools\netcoreapp3.1\any\appsettings.json'.
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.HandleException(ExceptionDispatchInfo info)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load(Boolean reload)
      at Microsoft.Extensions.Configuration.FileConfigurationProvider.Load()
      at Microsoft.Extensions.Configuration.ConfigurationRoot..ctor(IList`1 providers)
      at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
      at Lucene.Net.Cli.Program.Main(String[] args) in D:\a\1\s\src\dotnet\tools\lucene-cli\Program.cs:line 27
      

      Adding a text file named appsettings.json to the location specified in the error message with opening and closing brackets will prevent the exception.

      appsettings.json

      {
      }
      

      IMPORTANT: There must be at least opening and closing curly brackets in the file, or it won't be parsed as valid JSON.

    • J2N versions prior to version 2.0.0-beta-0012 had an infinite recursion bug on Xamarin.Android which caused fatal crashes in Lucene.NET. Upgrading J2N to 2.0.0-beta-0012 or higher will prevent these crashes from occurring.

    Change Log

    Bugs

    • #294 - Fixed the description for Lucene.NET 2.9.4 in the API documentation.

    Improvements

    • #260 - Changed culture-sensitivity of function-based queries to use the invariant culture when parsing/formatting values, to match the Lucene implementation.
    • #266 - Added documentation to show example usage of injecting custom codecs, doc values formats, and postings formats with pure DI or using a dependency injection container. An demo project named Lucene.Net.Tests.TestFramework.DependencyInjection has also been provided as an example of using Microsoft.Extensions.DependencyInjection to provide custom codecs to the test environment so they can be used during testing of other components.
    • #295 - PERFORMANCE Fixed boxing issue with NUnit asserts when passing common primitive types. This significantly improves the performance of running tests.
    • Upgraded to ICU4N 60.1.0-alpha.350 and refactored the collation features of Lucene.Net.ICU to utilize the new UCultureInfo class.
    • #282 - Restructured API documentation so each assembly has its own "mini-site" to fix x-ref linking issue, which fixes the TOC and breadcrumbs.
    • #282 - Documented the docs and website building procedures.
    • #282 - Fixed links on the download page and added missing packages (https://lucenenet.apache.org/download/version-4.8.0-beta00009.html).

    New Features

    • #254 - Implementation of "System Properties" which allow users to supply configuration values to the test framework and lucene-cli using configuration files or environment variables.
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00008(May 4, 2020)

    This release contains performance improvements and bug fixes.

    The Lucene.Net.Support namespace has been phased out, and most of the components that previously existed there have been refactored and moved to J2N, a library to fill in gaps in functionality between the JDK and .NET.

    Known Issues

    • J2N versions prior to version 2.0.0-beta-0012 had an infinite recursion bug on Xamarin.Android which caused fatal crashes in Lucene.NET. Upgrading J2N to 2.0.0-beta-0012 or higher will prevent these crashes from occurring.

    Change Log

    Breaking Changes

    • #239 - Changed Append() overloads of OpenStringBuilder to use startIndex/count rather than start/end to match the .NET convention

    • #241 - Made public fields into properties in ArabicStemmer

    • Lucene.Net.Collections: Factored out unmodifiable methods and related classes in favor of J2N's AsReadOnly() extension methods

    • SWEEP: Lucene.Net.Support: Factored out Equatable, EquatableList, and EquatableSet, and replaced with collections from J2N

    • SWEEP: Lucene.Net.Support: Factored out Collections.Equals() and Collections.GetHashCode()

    • Lucene.Net.Support.Collections: Factored out Swap() and Shuffle() in favor of J2N's implementation

    • Lucene.Net.Support: Factored out IdentityComparer, IdentityHashMap, and IdentityHashSet and used J2N.Runtime.CompilerServices.IdentityEqualityComparer in conjunction with standard Dictionary and HashSet types

    • Factored out TreeSet and TreeDictionary from Lucene.Net.Support in favor of J2N.Collections.Generic.SortedSet and J2N.Collections.Generic.SortedDictionary

    • Lucene.Net.Support.PriorityQueue: Factored out in favor of J2N's implementation

    • Lucene.Net.Support: Refactored ConcurrentHashSet into ConcurrentSet - a wrapper class that can be used to synchronize any set object (ordered or not), similar to how it was done in Java. Changed ordered concurrent set types back to the original type from Lucene.

    • Lucene.Net.Search.FieldComparer: Replaced CompareTo() calls with JCG.Comparer<T>.Default.Compare(), factoring out Lucene.Net.Support.SignedZeroComparer in the process.

    • Factored out Character, ICharSequence, StringBuilderCharSequenceWrapper, StringBuilderExtensions, StringCharSequenceWrapper, and most StringExtensions methods in favor of J2N's implementation

    • Changed semantics of CharTermAttribute.Append() overloads to act more like the .NET StringBuilder class.

      1. The 3rd parameter was changed from an exclusive end index to a count.
      2. A null parameter on a single parameter overload will be a no-op instead of appending the text "null".
      1. A null parameter on a 3 parameter overload will be a no-op if both startIndex and count are 0, otherwise it will throw an ArgumentNullException.
    • Lucene.Net.Support: Factored out GeneralKeyedCollection and AttributeItem in favor of J2N.Collections.Generic.LinkedDictionary

    • SWEEP: Upgraded to account for breaking changes (AsCharSequence() and BitOperation) in J2N.

    • Lucene.Net.Support.Collections: Factored out ImplementsGenericInterface() in favor of J2N's implementation

    • Lucene.Net.Support.Collections: Factored out Singleton() method and used collection initializers instead

    • Lucene.Net.Support.Collections: Removed NewSetFromMap method and related SetFromMap class

    • Lucene.Net.Support: Added ConcurrentHashSet class for internal use, and factored out the use ConcurrentSet throughout the project

    • Removed Lucene.Net.Support.ConcurrentHashMapWrapper from the public API and renamed ConcurrentDictionary

    • Marked ExceptionToClassNameConventionAttribute, ExceptionToNetNumericConventionAttribute, ExceptionToNullableEnumConventionAttribute, and WritableArrayAttribute internal

    • Factored out StringExtensions

    • Factored out DictionaryExtensions

    • Factored out LurchTable in favor of J2N's implementation

    • Lucene.Net.Support.Threading: Deleted ThreadLock, DisposableThreadLocalProfiler

    • Factored out Lucene.Net.Support.AssemblyExtensions in favor of J2N's implementation

    • Lucene.Net.Support: Marked all types internal

    • Moved Lucene.Net.Support.SystemConsole to Lucene.Net.Util namespace

    • Lucene.Net.Support: Moved ExceptionExtensions to Lucene.Net.Util namespace

    • Lucene.Net.Support.ListExtensions: Moved AddRange, Sort, TimSort, and IntroSort extension methods to Lucene.Net.Util.ListExtensions.

    • Lucene.Net.Support.NumberFormat: Moved to Lucene.Net.Util namespace

    • Lucene.Net.TestFramework.Support.JavaCompatibility.AbstractBeforeAfterRule: Moved from Lucene.Net.Support namespace to Lucene.Net.Util

    • Lucene.Net.TestFramework.Support: Changed namespace of ApiScanTestBase, CultureInfoSupport, and ExceptionSerializationTestBase to Lucene.Net.Util

    • Lucene.Net.Util.NumberFormat, Lucene.Net.QueryParsers.Flexible.Standard.Config.NumberDateFormat: Changed protected locale field to private, made property named Culture, and changed constructors and methods to use "culture" rather than "locale" for parameter names

    • Lucene.Net.Support: Moved SystemProperties class to Lucene.Net.Util namespace

    • Lucene.Net.Benchmark.Support: Moved EnglishNumberFormatExtensions to Lucene.Net.Util namespace

    Bugs

    • #237 - Fix duplicate FragNum value on Highlight.TextFragment
    • #235 - Fixed broken links on API documentation home page
    • Lucene.Net.Search.BooleanClause::Equals(BooleanClause): Fixed potential null reference exception when Query is set to null
    • LUCENENET-645 - Added missing call to FileStream::Flush() in FSIndexOutput::Flush() that was preventing persistence to disk from occuring at the necessary phase
    • SWEEP: Pass Random instance to all Shuffle() method calls to ensure the same psuedo-random sequence is used based on the seed
    • Lucene.Net.Index.DocumentsWriterDeleteQueue::DeleteSlice(): Removed extra Debug.Assert() statement that wasn't in Lucene which caused Lucene.Net.Index.TestIndexWriterUnicode::TestEmbeddedFFFF() test to fail when running in debug mode

    Improvements

    • #234 - Moved markdown converter to the src/docs folder
    • #235 - Updated API documentation so each release exists on its own URL
    • #235 - Updated documentation converter to be more robust when dealing with namespace matching
    • #239 - Replaced all private IComparer<T> implementations with the built in .NET Comparer.Create() factory method
    • #240 - Implemented IAppendable on OpenStringBuilder
    • PERFORMANCE - Removed unnecessary memory copy operations from various IndexReader dependencies
    • LUCENENET-640 - PERFORMANCE - Factored out WeakIdentityMap in favor of .NET's ConditionalWeakTable
    • SWEEP: Factored out LinkedHashMap in favor of J2N's LinkedDictionary
    • Lucene.Net.Tests.Search.TestFieldCache::Test(): Simplified expression with LINQ query
    • Lucene.Net.Highlighter, Lucene.Net.Tests.Spatial: Swapped in LinkedHashSet from J2N, like in the original Lucene implementation
    • SWEEP: Factored out Lucene.Net.Support.HashMap in favor of J2N.Collections.Generic.Dictionary
    • SWEEP: Factored out System.Collections.Generic.SortedDictionary in favor of J2N.Collections.Generic.SortedDictionary
    • Lucene.Net.Analysis.Kuromoji.Dict.UserDictionary: Swapped out C5 TreeDictionary for J2N.Collections.Generic.SortedDictionary
    • SWEEP: Changed System.Collections.Generic.SortedSet to J2N.Collections.Generic.SortedSet
    • SWEEP: Factored out C5's TreeSet
    • SWEEP: Swapped out System.Collections.Generic.HashSet for J2N.Collections.Generic.HashSet
    • SWEEP: Factored out Arrays.AsList(), which was causing both additional operational complexity and unnecessary memory allocations
    • Lucene.Net.Util.Automaton.State: Implemented IEquatable<State>, changed enumerator to a struct
    • Removed FEATURE_HASHSET_CAPACITY, since J2N now has the full .NET Core 3.x implementation with a capacity constructor
    • Lucene.Net.Util.Fst.ListOfOutputs::Merge(): Streamlined so we don't have so many casts
    • Lucene.Net.Util.Fst: Use J2N.Collections.List<T> for the closing type of Outputs<T> to ensure the outputs can be compared for structural equality
    • Lucene.Net.TestFramework.Search.AssertingScorer: Changed to use ConditionalWeakTable/WeakDictionary
    • LUCENENET-610, LUCENENET-640 - Reduced locking in FieldCache and its dependent classes
    • Added cross-framework compatibility for nullable attributes
    • Changed project files to automatically generate InternalsVisibleTo attributes based on ItemGroup/InternalsVisibleTo elements
    • Lucene.Net.Support.DictionaryExtensions: Optimized Put() method, added guard clauses to Put and PutAll
    • Lucene.Net.Support.DictionaryExtensions: Factored out Load() and Store() methods in favor of J2N's implementation
    • LUCENENET-642 - Lucene.Net.Analysis.TokenStream: Removed Reflection code that is used to force the end user to make TokenStream subclasses or their IncrementToken() method sealed
    • LUCENENET-643 - PERFORMANCE - Lucene.Net.Support.IO.FileStreamExtensions::Read(): Moved to StreamExtensions class and optimized to read bytes in bulk instead of one byte at a time
    • Lucene.Net.Tests.Index (TestBagOfPositions + TestBagOfPostings): Fixed performance issue due to converting int to string and string to int in the current culture
    • Lucene.Net.Store (FSDirectory + BufferedIndexOutput): Refactored FSDirectory.FSIndexOutput to utilize the FileStream buffer only, rather than using both a FileStream buffer and the buffer in BufferedIndexOutput.

    New Features

    • LUCENENET-642 - Added code analyzer and code fix to force the end user to make TokenStream subclasses or their IncrementToken() method sealed at design time, rather than running Reflection code at runtime to make this check
    Source code(tar.gz)
    Source code(zip)
  • Lucene.Net_4_8_0_beta00007(Jan 29, 2020)

    This release contains impactful performance improvements and bug fixes.

    NOTE: The Lucene.Net.Support namespace in the core Lucene.Net assembly is being phased out for external use. Much of what used to be in this namespace has been made into first-class components and moved to J2N, a library to fill in gaps in functionality between the JDK and .NET. Please do not add any new dependencies on Lucene.Net.Support in the core Lucene.Net library. Also, please open a new JIRA ticket if you have a dependency on a component from Lucene.Net.Support that is not available in J2N.

    This release drops support for .NET Standard 1.6 and adds support for .NET Standard 2.1.

    Change Log

    Breaking Changes

    • Lucene.Net.Codecs (in Lucene.Net library) - Changed the following methods to properties with the same name for API consistency
      • Lucene.Net.Codecs.DefaultCodecFactory::AvailableServices()
      • Lucene.Net.Codecs.DefaultDocValuesFormatFactory::AvailableServices()
      • Lucene.Net.Codecs.DefaultPostingsFormatFactory::AvailableServices()
      • Lucene.Net.Codecs.Codec::AvailableCodecs()
      • Lucene.Net.Codecs.DocValuesFormat::AvailableDocValuesFormats()
      • Lucene.Net.Codecs.PostingsFormat::AvailablePostingsFormats()
    • Lucene.Net.Analysis.Kuromoji - Changed the following methods to properties named Instance for API consistency
      • Lucene.Net.Analysis.Kuromoji.Dict.CharacterDefinition::GetInstance()
      • Lucene.Net.Analysis.Kuromoji.Dict.ConnectionCosts::GetInstance()
      • Lucene.Net.Analysis.Kuromoji.Dict.TokenInfoDictionary::GetInstance()
      • Lucene.Net.Analysis.Kuromoji.Dict.UnknownDictionary::GetInstance()
    • Lucene.Net.Support.Collections::AddAll() - Factored out in favor of ISet<T>.UnionWith()
    • Lucene.Net.Support.IdentityComparer::ctor() - Factored out in favor of new Default static property
    • Lucene.Net.Support.DictionaryExtensions::EntrySet() - Factored out because IDictionary<TKey, TValue> is already enumerable and copying its contents to another data structure for the purpose of enumeration is wasteful and unnecessary
    • Lucene.Net.Support.SetExtensions::AddAll() - Factored out in favor of UnionWith(), Lucene.Net.Support.DictionaryExtensions.PutAll(), or AddRange() depending on collection type
    • Lucene.Net.Support.HashMap<TKey, TValue> - Changed behavior of indexer property to throw a KeyNotFoundException if the key doesn't exist. The exception can already be avoided by using the TryGetValue() method.
    • Lucene.Net.Support.Buffer - Factored out in favor of J2N.IO.Buffer
    • Lucene.Net.Support.ByteBuffer - Factored out in favor of J2N.IO.ByteBuffer
    • Lucene.Net.Support.LongBuffer - Factored out in favor of J2N.IO.LongBuffer
    • Lucene.Net.Support.Number - Factored out the following methods in favor of their counterparts in J2N
      • Signum() > J2N.MathExtensions::Signum()
      • SingleToInt32Bits() > J2N.BitConversion::SingleToInt32Bits()
      • SingleToRawInt32Bits() > J2N.BitConversion::SingleToRawInt32Bits()
      • Int64BitsToDouble() > J2N.BitConversion::Int64BitsToDouble()
      • DoubleToInt64Bits() > J2N.BitConversion::DoubleToInt64Bits()
      • DoubleToRawInt64Bits() > J2N.BitConversion::DoubleToRawInt64Bits()
      • ToBinaryString() > J2N.IntegralNumberExtensions::ToBinaryString()
      • URShift() > J2N.Numerics.BitOperationExtensions::TripleShift()
      • ToString(long, radix) > J2N.IntegralNumberExtensions::ToString(long, radix)
      • BitCount() > J2N.Numerics.BitOperationExtensions::PopCount()
      • NumberOfLeadingZeros() > J2N.IntegralNumberExtensions::LeadingZeroCount()
      • NumberOfTrailingZeros() > J2N.IntegralNumberExtensions::TrailingZeroCount()
      • RotateLeft() > J2N.IntegralNumberExtensions::RotateLeft()
      • RotateRight() > J2N.IntegralNumberExtensions::RotateRight()
    • Lucene.Net.Support.Number - Removed the following methods
      • FlipEndian()
      • IsNumber()
      • ToInt64()
    • Lucene.Net.Support.MathExtensions - Factored out the following methods in favor of their counterparts in J2N
      • ToDegrees() > J2N.MathExtensions::ToDegrees()
      • ToRadians() > J2N.MathExtensions::ToRadians()
    • Lucene.Net.Support.AtomicBoolean - Factored out in favor of J2N.Threading.Atomic.AtomicBoolean
    • Lucene.Net.Support.AtomicInt32 - Factored out in favor of J2N.Threading.Atomic.AtomicInt32
    • Lucene.Net.Support.AtomicInt64 - Factored out in favor of J2N.Threading.Atomic.AtomicInt64
    • Lucene.Net.Support.AtomicObject - Factored out in favor of J2N.Threading.Atomic.AtomicReference
    • Lucene.Net.Support.AtomicReferenceArray - Factored out in favor of J2N.Threading.Atomic.AtomicReferenceArray
    • Lucene.Net.Support.Threading.ThreadClass - Factored out in favor of J2N.Threading.ThreadJob
    • Lucene.Net.Support.Character - Factored out the following methods in favor of their counterparts in J2N
      • Digit()
      • ForDigit()
    • Lucene.Net.Support.StringTokenizer - Factored out in favor of J2N.Text.StringTokenizer
    • Lucene.Net.Support.CultureContext - Factored out in favor of J2N.Globalization.CultureContext
    • Lucene.Net.Support.IndexWriterConfigExtensions - Moved to Lucene.Net.Index.Extensions namespace
    • Lucene.Net.Documents.IndexableFieldExtensions - Moved to Lucene.Net.Documents.Extensions namespace
    • Lucene.Net.Documents.DocumentExtensions - Moved to Lucene.Net.Documents.Extensions namespace
    • Lucene.Net.Support.IResourceManagerFactory - Moved to Lucene.Net.Util namespace
    • Lucene.Net.Support.BundleResourceMangerFactory Moved to Lucene.Net.Util namespace

    Bugs

    • LUCENENET-615 - PerFieldAnalyzerWrapper and PerFieldReuseStrategy must support null keys.
    • LUCENENET-617 - Reverted changes introduced in pull #222 that caused deadlock in Lucene.Net.Tests.Replicator.IndexAndTaxonomyReplicationClientTest::TestConsistencyOnExceptions().
    • Lucene.Net.Tests.Analysis.Common/Analysis/Util/TestCharArrayMap_TestCharArrayMap() - Was failing in Turkish, lowercasing must be done in the invariant culture to match Lucene
    • Lucene.Net.Tests.Util.Fst.TestFSTs::TestPrimaryKeys() - Fixed sorting issue that was causing the test to fail with negative values
    • SWEEP: Corrected number-to-string and string-to-number conversions to use the invariant culture, as was the case in Lucene
    • LUCENENET-622 - Lucene.Net.Tests.Util.TestVersionComparer::TestVersions() - Fixed version number conversion to use the invariant culture so the test consistently passes
    • LUCENENET-621 - Lucene.Net.Tests.Search.TestSearchAfter::TestQueries() - Fixed number-to-string conversion to use the invariant culture so the test consistently passes
    • `Lucene.Net.Tests.Benchmark.ByTask.Tasks.WriteLineDocTaskTest::TestMultiThreaded(): Added lock to synchronize write files. Also refactored test to assist future debugging efforts.
    • Lucene.Net.Analysis.Common - Fixed to use invariant culture converting numbers and uppercasing/lowercasing
    • Lucene.Net.Benchmark - Fixed to use invariant culture converting numbers and uppercasing/lowercasing
    • Lucene.Net.Facet - Fixed to use invariant culture for number conversion
    • Lucene.Net.Misc - Fixed to use invariant culture for number conversion
    • Lucene.Net.Suggest - Fixed to use invariant culture for number conversion
    • Lucene.Net.Grouping - Fixed to use invariant culture for number conversion
    • Lucene.Net.Spatial - Fixed to use invariant culture for number conversion
    • Lucene.Net.Tests.Replicator - Fixed issue with incorrect error message due to missing Dispose() call
    • Lucene.Net.Tests.Replicator.LocalReplicatorTest::TestObtainMissingFile() - Added missing catch block for DirectoryNotFoundException
    • Lucene.Net.Support.Codecs - Fixed initialization locking between parallel tasks. Also added guard clauses and inlined variable declarations.
    • #154 - Fix OpenBitSet.Union and .Xor methods.
    • LUCENENET-618 - CRITICAL: Fixed Lucene.Net.Store.NativeFSLockFactory to be thread-safe on non-Windows operating systems
    • Fixed broken OS detection on .NET Framework
    • LUCENENET-602 - Fixed generic stucts of LurchTable to function on Xamarin.iOS without throwing exceptions
    • Lucene.Net.TestFramework.Index.ThreadedIndexingAndSearchingTestCase - WeakDictionary<TKey, TValue> must be wrapped to make it thread safe
    • Lucene.Net.Codecs.Lucene45.Lucene45DocValuesConsumer::MISSING_ORD - Correct value should be -1L to match Lucene

    Improvements

    • azure-pipelines.yml: Added job to generate documentation
    • Improved automated documentation converter accuracy
    • Moved Lucene.Net.Util.VirtualMethod to its original location in the Lucene.Net assembly
    • Updated usage examples on home page of web site
    • Changed categorization on API documentation to list by assembly rather than namespace
    • Lucene.Net.Analysis.Phonetic - Create culture during static initialization to improve performance
    • Lucene.Net.Index.CheckIndex - Added guard clause to ensure dir is not null
    • Lucene.Net.Support.IO.FileSupport::CreateTempFile() - Simplified error handling to use a when clause to only catch the relevant exceptions (improving thread safety)
    • SWEEP: CA1810 - Avoid static constructors (see #224 (comment)).
    • LUCENENET-612 & LUCENENET-615 - Added documentation to clarify that the IDictionary<TKey, TValue> type that is provided will determine how the component behaves.
    • Lucene.Net.Support.BitArrayExtensions::Cardinality() - Replaced implementation with one that benchmarked more than 3x faster, at the cost of a small amount of RAM
    • Consolidated build features for the solution in the root Directory.Build.targets file
    • Added target for .NET Standard 2.1
    • Removed target for .NET Standard 1.6
    • Removed support code for targeting .NET Framework 3.5
    • LUCENENET-610 - Added conditional compilation to favor WeakConditionalTable<TKey, TValue> over WeakDictionary<TKey, TValue> when the features Lucene requires are available.
    • Lucene.Net.Support.Arrays - Changed Equals() and GetHashCode() to be backed by J2N.Collections.ArrayEqualityComparer<T>.OneDimensional for improved performance
    • Lucene.Net.Support.Arrays::ToString() - Changed implementation to the same as Apache Harmony for improved performance
    • Lucene.Net.Support.Character
      • ToChars() - Cascade to J2N.Character.ToChars(), which benchmarked 5x faster
      • ToUpper() - Cascade to J2N.Character.ToUpper()
      • ToLower() - Cascade to J2N.Character.ToLower()
      • IsLetter() - Cascade to J2N.Character.IsLetter()
    • Lucene.Net.Support.DictionaryExtensions::Load() - Cascade the call to J2N.PropertyExtensions::LoadProperties()
    • Lucene.Net.Support.DictionaryExtensions::Store() - Cascade the call to J2N.PropertyExtensions::SaveProperties()
    • Upgraded lucene-cli to target netcoreapp3.1
    • Fixed NuGet Warning NU5048, deprecation of PackageIconUrl element
    • build.ps1 - Added testname/framework to test script to show which projects are running
    • Changed testing target from net451 to net48
    • Changed testing target from netcoreapp2.1 to netcoreapp2.2
    • Added testing target for netcoreapp3.1
    • Added new debug artifact for .pdb files so they aren't pushed into our Apache release distribution binary
    • Added .rat-excludes file for Apache RAT

    New Features

    • LUCENENET-614 - Released Lucene.Net.TestFramework on NuGet
    • Added Lucene.Net.Tests.TestFramework (ported compatible tests from Lucene 8.2.0, since Lucene 4.8.0 did not have any tests)
    • Lucene.Net.Codecs.DefaultCodecFactory - Added CustomCodecTypes property that can be used to initialize custom codec types without requiring a subclass
    • Lucene.Net.Codecs.DefaultDocValuesFormatFactory - Added CustomDocValuesFormatTypes property that can be used to initialize custom codec types without requiring a subclass
    • Lucene.Net.Codecs.DefaultPostingsFormatFactory - Added CustomPostingsFormatTypes property that can be used to initialize custom codec types without requiring a subclass
    • LUCENENET-623 - Released Lucene.Net.Analysis.OpenNLP on NuGet
    • LUCENENET-568 - Released Lucene.Net.Analysis.Morfologik on NuGet
    Source code(tar.gz)
    Source code(zip)
Owner
The Apache Software Foundation
The Apache Software Foundation
Apache Lucene is a high-performance, full featured text search engine library written in Java.

Apache Lucene is a high-performance, full featured text search engine library written in Java.

The Apache Software Foundation 74 Mar 27, 2021
Apache Lucene and Solr open-source search software

Apache Lucene and Solr have separate repositories now! Solr has become a top-level Apache project and main line development for Lucene and Solr is hap

The Apache Software Foundation 4.1k Mar 11, 2021
A proof-of-concept serverless full-text search solution built with Apache Lucene and Quarkus framework.

Lucene Serverless This project demonstrates a proof-of-concept serverless full-text search solution built with Apache Lucene and Quarkus framework. ✔️

Arseny Yankovsky 17 Mar 29, 2021
Apache Solr is an enterprise search platform written in Java and using Apache Lucene.

Apache Solr is an enterprise search platform written in Java and using Apache Lucene. Major features include full-text search, index replication and sharding, and result faceting and highlighting.

The Apache Software Foundation 46 Mar 27, 2021
Free and Open, Distributed, RESTful Search Engine

Elasticsearch A Distributed RESTful Search Engine https://www.elastic.co/products/elasticsearch Elasticsearch is a distributed RESTful search engine b

elastic 54k Mar 13, 2021
ElasticSearch SQL查询适配器,为Elasticsearch提供SQL查询的功能。支持跨集群查询,可拓展异构数据源。将calcite基于配置文件的配置方式改为通过代码灵活控制,可动态增加数据源,并支持完整的SQL查询功能。

?? 使用指南 ElasticSearchSQL查询适配器,为Elasticsearch提供SQL查询的功能。支持完整的SQL查询和跨Elasticsearch集群查询。通过代码简单配置,使项目根据Elasticsearch的index映射为Table。 License CodeBeat Langu

弘树丶 17 Mar 20, 2021
OpenSearch is an open source distributed and RESTful search engine.

OpenSearch is an open source search and analytics engine derived from Elasticsearch

null 1.5k Apr 15, 2021