How to delete all data from solr and hbase

SolrHbase

Solr Problem Overview

How do I delete all data from solr by command? We are using solr with lily and hbase.

How can I delete data from both hbase and solr?

http://lucene.apache.org/solr/4_10_0/tutorial.html#Deleting+Data

Solr Solutions

Solution 1 - Solr

If you want to clean up Solr index -

you can fire http url -

http://host:port/solr/[core name]/update?stream.body=<delete><query>*:*</query></delete>&commit=true

(replace [core name] with the name of the core you want to delete from). Or use this if posting data xml data:

<delete><query>*:*</query></delete>

Be sure you use commit=true to commit the changes

Don't have much idea with clearing hbase data though.

Solution 2 - Solr

I've used this request to delete all my records but sometimes it's necessary to commit this.

For that, add &commit=true to your request :

http://host:port/solr/core/update?stream.body=<delete><query>*:*</query></delete>&commit=true

Solution 3 - Solr

Post json data (e.g. with curl)

curl -X POST -H 'Content-Type: application/json' \
    'http://<host>:<port>/solr/<core>/update?commit=true' \
    -d '{ "delete": {"query":"*:*"} }'

Solution 4 - Solr

You can use the following commands to delete. Use the "match all docs" query in a delete by query command: 

'<delete><query>*:*</query></delete>

You must also commit after running the delete so, to empty the index, run the following two commands:

curl http://localhost:8983/solr/update --data '<delete><query>*:*</query></delete>' -H 'Content-type:text/xml; charset=utf-8'
curl http://localhost:8983/solr/update --data '<commit/>' -H 'Content-type:text/xml; charset=utf-8'

Another strategy would be to add two bookmarks in your browser:

http://localhost:8983/solr/update?stream.body=<delete><query>*:*</query></delete>
http://localhost:8983/solr/update?stream.body=<commit/>


Source docs from SOLR:
https://wiki.apache.org/solr/FAQ#How_can_I_delete_all_documents_from_my_index.3F

Solution 5 - Solr

If you want to delete all of the data in Solr via SolrJ do something like this.

public static void deleteAllSolrData() {
    HttpSolrServer solr = new HttpSolrServer("http://localhost:8080/solr/core/");
    try {
      solr.deleteByQuery("*:*");
    } catch (SolrServerException e) {
      throw new RuntimeException("Failed to delete data in Solr. "
          + e.getMessage(), e);
    } catch (IOException e) {
      throw new RuntimeException("Failed to delete data in Solr. "
          + e.getMessage(), e);
    }
}

If you want to delete all of the data in HBase do something like this.

public static void deleteHBaseTable(String tableName, Configuration conf) {
    HBaseAdmin admin = null;    
    try {
        admin = new HBaseAdmin(conf);
        admin.disableTable(tableName);
        admin.deleteTable(tableName);
    } catch (MasterNotRunningException e) {
        throw new RuntimeException("Unable to delete the table " + tableName
        + ". The actual exception is: " + e.getMessage(), e);
    } catch (ZooKeeperConnectionException e) {
        throw new RuntimeException("Unable to delete the table " + tableName
        + ". The actual exception is: " + e.getMessage(), e);
    } catch (IOException e) {
        throw new RuntimeException("Unable to delete the table " + tableName
        + ". The actual exception is: " + e.getMessage(), e);
    } finally {
        close(admin);
    }
 }

Solution 6 - Solr

Use the "match all docs" query in a delete by query command: :

You must also commit after running the delete so, to empty the index, run the following two commands:

curl http://localhost:8983/solr/update --data '<delete><query>*:*</query></delete>' -H 'Content-type:text/xml; charset=utf-8'

curl http://localhost:8983/solr/update --data '<commit/>' -H 'Content-type:text/xml; charset=utf-8'

Solution 7 - Solr

I came here looking to delete all documents from solr instance through .Net framework using SolrNet. Here is how I was able to do it:

Startup.Init<MyEntity>("http://localhost:8081/solr");
ISolrOperations<MyEntity> solr =
    ServiceLocator.Current.GetInstance<ISolrOperations<MyEntity>>();
SolrQuery sq = new SolrQuery("*:*");
solr.Delete(sq);
solr.Commit();

This has cleared all the documents. (I am not sure if this could be recovered, I am in learning and testing phase of Solr, so please consider backup before using this code)

Solution 8 - Solr

From the command line use:

 bin/post -c core_name -type text/xml -out yes -d $'<delete><query>*:*</query></delete>'

Solution 9 - Solr

fire this in the browser

http://localhost:8983/solr/update?stream.body=<delete><query>*:*</query></delete>&commit=true this commmand will delete all the documents in index in solr

Solution 10 - Solr

I've used this query to delete all my records.

http://host/solr/core-name/update?stream.body=%3Cdelete%3E%3Cquery%3E*:*%3C/query%3E%3C/delete%3E&commit=true

Solution 11 - Solr

The curl examples above all failed for me when I ran them from a cygwin terminal. There were errors like this when i ran the script example.

curl http://192.168.2.20:7773/solr/CORE1/update --data '<delete><query>*:*</query></delete>' -H 'Content-type:text/xml; charset=utf-8'
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int></lst>
</response>
<!-- 
     It looks like it deleted stuff, but it did not go away
     maybe because the committing call failed like so 
-->
curl http://192.168.1.2:7773/solr/CORE1/update --data-binary '' -H 'Content-type:text/xml; charset=utf-8'
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">400</int><int name="QTime">2</int></lst><lst name="error"><str name="msg">Unexpected EOF in prolog
 at [row,col {unknown-source}]: [1,0]</str><int name="code">400</int></lst>
</response>

I needed to use the delete in a loop on core names to wipe them all out in a project.

This query below worked for me in the Cygwin terminal script.

curl http://192.168.1.2:7773/hpi/CORE1/update?stream.body=<delete><query>*:*</query></delete>&commit=true
<?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">1</int></lst>
</response>

This one line made the data go away and the change persisted.

Solution 12 - Solr

I tried the below steps. It works well.

  • Please make sure the SOLR server it running

  • Just click the link Delete all SOLR data which will hit and delete all your SOLR indexed datas then you will get the following details on the screen as output.

    <response>
  <lst name="responseHeader">
    <int name="status">0</int>
    <int name="QTime">494</int>
  </lst>
</response>

  • if you are not getting the above output then please make sure the following.

  • I used the default host (localhost) and port (8080) on the above link. please alter the host and port if it is different in your end.

  • The default core name should be collection / collection1. I used collection1 in the above link. please change it too if your core name is different.

Solution 13 - Solr

To delete all documents of a Solr collection, you can use this request:

curl -X POST -H 'Content-Type: application/json' --data-binary '{"delete":{"query":"*:*" }}' http://localhost:8983/solr/my_collection/update?commit=true

It uses the JSON body.

Solution 14 - Solr

If you need to clean out all data, it might be faster to recreate collection, e.g.

solrctl --zk localhost:2181/solr collection --delete <collectionName>
solrctl --zk localhost:2181/solr collection --create <collectionName> -s 1

Solution 15 - Solr

I made a JavaScript bookmark which adds the delete link in Solr Admin UI

javascript: (function() {
    var str, $a, new_href, href, upd_str = 'update?stream.body=<delete><query>*:*</query></delete>&commit=true';
    $a = $('#result a#url');
    href = $a.attr('href');
    str = href.match('.+solr\/.+\/(.*)')[1];
    new_href = href.replace(str, upd_str);
    $('#result').prepend('<a id="url_upd" class="address-bar" href="' + new_href + '"><strong>DELETE ALL</strong>   ' + new_href + '</a>');
})();

enter image description here

Solution 16 - Solr

If you're using Cloudera 5.x, Here in this documentation is mentioned that Lily maintains the Real time updations and deletions also.

Configuring the Lily HBase NRT Indexer Service for Use with Cloudera Search

> As HBase applies inserts, updates, and deletes to HBase table cells, > the indexer keeps Solr consistent with the HBase table contents, using > standard HBase replication.

Not sure iftruncate 'hTable' is also supported in the same.

Else you create a Trigger or Service to clear up your data from both Solr and HBase on a particular Event or anything.

Solution 17 - Solr

When clearing out a Solr index, you should also do a commit and optimize after running the delete-all query. Full steps required (curl is all you need): http://www.alphadevx.com/a/365-Clearing-a-Solr-search-index

Solution 18 - Solr

Solr I am not sure but you can delete all the data from hbase using truncate command like below:

truncate 'table_name'

It will delete all row-keys from hbase table.

Categories
Recommended Solr Solutions
Recommended Hbase Solutions

Previous Solution:

Benchmarking (python vs. c++ using BLAS) and (numpy)

C++PythonNumpyBenchmarkingBlas

Next Solution:

How can I truncate a double to only two decimal places in Java?

JavaDoubleTruncateDecimalformat

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionXMenView Question on Stackoverflow
Solution 1 - SolrJayendraView Answer on Stackoverflow
Solution 2 - SolrShowtim3View Answer on Stackoverflow
Solution 3 - SolrFrank R.View Answer on Stackoverflow
Solution 4 - SolrNavView Answer on Stackoverflow
Solution 5 - SolrRATaboraView Answer on Stackoverflow
Solution 6 - SolrNanhe KumarView Answer on Stackoverflow
Solution 7 - SolrHabibView Answer on Stackoverflow
Solution 8 - SolrMurtaza ManasawalaView Answer on Stackoverflow
Solution 9 - SolrbittuView Answer on Stackoverflow
Solution 10 - SolrSuf_MalekView Answer on Stackoverflow
Solution 11 - SolrndasusersView Answer on Stackoverflow
Solution 12 - SolrGanesa VijayakumarView Answer on Stackoverflow
Solution 13 - SolryouhansView Answer on Stackoverflow
Solution 14 - SolrTagarView Answer on Stackoverflow
Solution 15 - SolrMyroslavNView Answer on Stackoverflow
Solution 16 - SolrMurtaza KanchwalaView Answer on Stackoverflow
Solution 17 - SolralphadevxView Answer on Stackoverflow
Solution 18 - SolrKapilView Answer on Stackoverflow