acts_as_ferret will_paginate

Update: This is not needed with recent versions of acts_as_ferret.

Here’s a little nugget to add to acts_as_ferret to make your searches paginate with will_paginate.

module ActsAsFerret
  module ClassMethods
    def paginate_search(query, options = {})
      page, per_page, total = wp_parse_options(options)
      pager = WillPaginate::Collection.new(page, per_page, total)
      options.merge!(:offset => pager.offset, :limit => per_page)
      result = find_by_contents(query, options)
      returning WillPaginate::Collection.new(page, per_page, result.total_hits) do |pager|
        pager.replace result
      end
    end
  end
end

Updated from Behrang’s comment based on changes to will_paginate.

There was a slight challenge in that will_paginate expects that you do one query to get the count, create a new collection object based on that count, and then perform the actual search. But acts_as_ferret does it all in one method call, so I have to create a temporary collection object to get the offset, then do the search and create the collection object. It’s a little messier than it needs to be, but it works.

Product.paginate_search params[:q], :page => params[:page]

ferret, pagination, plugin, rails, ruby, and search August 17, 2007

49 Comments

Adam Roth August 17, 2007

Very niiiice.
rudionrails August 23, 2007

Hi there,

I am not 100% sure now, but I think there’s a flaw in your code. This pagination will never appear.

You pass the offset and limit to the find_by_contents method, so the total_hits will never be above the per_page value. Hence, no pagination after all.
You need to run the find_by_contents without offset and limit first to get the “true” total_hits for your search. Then you can paginate.

The way to go is running the ferret search two times. Once to get the total_hits and once again to get the “real” search results with offset and limit for the current page. Not very good for performance, but I haven’t found a better way yet.

Let me know if I talk rubbish ;-)
Cheers, Rudi
Brandon August 23, 2007

rudionrails,

You talk rubbish! ;)

I know the code above works because I’m using it in production. total_hits on the result object returned by find_by_contents isn’t the number of results returned, but the number of hits in the ferret index. So even if you limit the results returned (which acts_as_ferret limits to 10 by default), the total_hits will still give you the total count.
rudionrails August 24, 2007

Hi again,

I’ve just double checked and my ferret search returns the total_hits like I explained above.

I have 3 test records in my DB. Using find_by_contents(“”) returns all three as expected. Using find_by_contents(“”, { :limit => 1 }) returns only 1 record (kind of as expected, too). I use ferret 0.11.4, perhaps there lies a difference? Or am I doing something wrong?

Rudi
rudionrails August 24, 2007

Oops I just noticed that textile messed up the message a little :-S
I did not mean to make things bold. There is a star between the find_by_contents quotes to get all records :-)

Brandon August 24, 2007

rudionrails,

That’s not what I’m seeing.

$ gem list ferret
ferret (0.11.4, 0.11.3)
$ script/console
&gt;&gt; result = Product.find_by_contents('*')
=&gt; #&lt;ActsAsFerret::SearchResults:0x32c97ac …&gt;
&gt;&gt; result.size
=&gt; 10
&gt;&gt; result.total_hits
=&gt; 326
&gt;&gt; result = Product.find_by_contents('*', :limit =&gt; 1)
=&gt; #&lt;ActsAsFerret::SearchResults:0x3125c20 …&gt;
&gt;&gt; result.size
=&gt; 1
&gt;&gt; result.total_hits
=&gt; 326

Tim August 30, 2007

Really like your lib and it works great except for one thing…

I seem to be getting an error when I can .total_hits – I get a WillPaginate::Collection error. And I know that this is because now we call:

Model.paginate_search params[:search],
:page => params[:page],
:per_page => 20

In my old app, I was doing an unless .total_hits == 0 do – “.total_hits whatevers matched your query.”

I can’t seem to get another .total_hits working :( – any ideas?

Thanks.
Brandon August 30, 2007

Tim,

The WillPaginate::Collection has a total_entries method, along with current_page, per_page. Those should get you what you want.
tarsolya August 30, 2007

Brandon,

just a quick question without trying this out (forgive me please on this one), but does this support :conditions on ferret’s find_by_contents?

Thanks,
András
tarsolya August 30, 2007

I mean, conditions.

Somehow it was stripped out from my previous comment.
Brandon August 30, 2007

tarsolya,

This should support anything that find_by_contents supports. All I’m doing is wrapping that to make it compatible with find_by_contents.

You may want to be careful though with :conditions, as the acts_as_ferret documentation states, because the conditions are applied to the search results from the ferret index, so your search results will most likely have fewer results that you expect per_page.
James H. September 5, 2007

Silly question, but are you dropping this code directly into your local copy of the AAF plugin, or are you injecting it some how?
Brandon September 5, 2007

James H., I’m putting it in the lib directory for my app and requiring it in environment.rb
Skyblaze September 6, 2007

i get this error:

undefined method `find_search’ for Customer:Class
Brandon September 6, 2007

Skyblaze,

Sounds like the method above isn’t being required. You’ll need to put a require statement in environment.rb if you put this snippet in the lib directory.
Skyblaze September 12, 2007

Ok now it works but i have a strange problem. If i do a search on two text fields all works correctly but if i do a simple search (for the customers table) as : “user_id:1” it returns an empty array. It seems it doesn’t want to work only with this field! How can it be possible?
Brandon September 12, 2007

Skyblaze:

Does find_by_contents return any results? If not, then there’s something wrong with your acts_as_ferret configuration and you should look to the acts_as_ferret documentation for more help.
claudia September 12, 2007

I had the same issue as rudionrails, with the search results paginating through the same ten results over and over again. I created an offset variable which is equal to per_page * (current_page – 1). This fixed the pagination for me, and now it works perfectly.
Skyblaze September 13, 2007

I tried different fields but with the user_id field it returns nothing and i don’t understand why. In my model file i don’t have the user_id field listed in “acts_as_ferret :fields => [:rag_soc]” (i’ve only that rag_soc field), but that doesn’t matter right? I can then use in my search string the form “field_name:value” right? But it doesn’t work!
Skyblaze September 13, 2007

actually i’ve tried to search other fields with that form but it doesn’t work i can only search in the default field
Brandon September 13, 2007

claudia: I’ll look into it. Maybe different versions of ferret/acts_as_ferret respond differently to the offset.

Skyblaze: Sorry, but this isn’t a support forum for acts_as_ferret. Any fields that you want to search with ferret need to be declared with the :feilds option to acts_as_ferret. Please go to the acts_as_ferret site and documentation unless you have specific issues with the code in this post.
EmmanuelOga September 13, 2007

For total compatibilitie with my previous acts_as_ferret code I added the total_hits method based on total_entries. Now it works seamlessly with all my code. Thank you very much!

module ActsAsFerret
module ClassMethods

def paginate_search(query, options = {})
options, page, per_page = wp_parse_options!(options)
pager = WillPaginate::Collection.new(page, per_page, nil)
options.merge!(:offset => pager.offset, :limit => per_page)
result = result = find_by_contents(query, options)
returning WillPaginate::Collection.new(page, per_page, result.total_hits) do |final_pager|
final_pager.replace result
def final_pager.total_hits
total_entries()
end
end
end
end

end
Skyblaze September 19, 2007

i noticed a strange behaviour. Your modified method to use will_paginate with acts_as_ferret worked for me perfectly then suddenly in a model the pagination doesn’t work correctly. If i put 2 as a value to the :per_page parameter (as i did before) it puts only one result per page creating 2 pages. If i put 1 as the :per_page paramter it puts 1 result per page but it creates 3 pages (for only two results) with the first page empty. On the other model(controller with exactly the same code all work perfectly. If i use the vanilla “paginate” will_paginate plugin method all work great!
Jd September 23, 2007

I wanted to do a search, and filter it with :find_conditions. I had to modify your code a little bit to make it work.

module ActsAsFerret
module ClassMethods
def paginate_search(query, options = {}, find_options = {})
options, page, per_page = wp_parse_options!(options)
pager = WillPaginate::Collection.new(page, per_page, nil)
options.merge!(:offset => pager.offset, :limit => per_page)
result = result = find_by_contents(query, options, find_options)
returning WillPaginate::Collection.new(page, per_page, result.total_hits) do |pager|
pager.replace result
end
end
end
end

And, in my controller

@posts = Post.paginate_search
params[:q],
{:per_page => 5, :page => params[:page]},
{:conditions => "some_condition"}
Brendon November 7, 2007

One thing this (or any other pagination system that I could find) doesn’t take into account is whether a particular logged in user (for example) can see certain records. They will all be in the ferret database (except for those that should never be in there, which we can define in ferret_enabled? in the model) but depending on if the user is allowed to see the results, we need to remove them from the results. Previously I tried this:
```
    @results.delete_if { |result|
      !result.component_instance.viewable?(user) &amp;&amp; (result.parent.respond_to?('password') &amp;&amp; result.parent.component_instance.password(session, user))
    }
```
(just a few conditions to decide whether to delete the object or not). This works well, but makes the pagination inaccurate as it removes items after the pagniator has done its magic on the find_by_contents method. A page with 5 results normally, might only show 4. If the last page had 1 result on it but this method deleted it then the page would be strangely empty etc…

Is there a way to filter the results before they’re paginated? I tried to do it inside the module but once the deleteif had run, the paginator complained that it couldn’t find total_hits in the array called result.

Your help would very much be appreciated :)

Just to clarify, the above filters aren’t able to be done via SQL or anything like that. They rely on being able to have the object in the applications context.
Brandon November 7, 2007

Brendon,

I think your best bet would be to push the requirements for who can view the models into the index. For example, if your model is only viewable by certain roles, add a #viewable_by method to your model that returns an array of the roles, and index this field. Then when you search, you could only search for objects that are viewable by the current user’s roles.

Other than that, I don’t have any ideas about how you would accomplish this without breaking pagination.
Brendon November 8, 2007

Thanks Brandon (cool name :) )

That’s a good idea, unfortunately it won’t work in my situation as one of the ways we protect content is via a simple password on a folder that contains the content. If the user has visited that folder before, they will have already been asked for a password and we would have stored a session variable to that effect, and that’s the only way we’d know, so we can’t store this in the index.

Thanks for your help however, and I think I might have had an idea (though I’m not near my workstation to check it) but it goes like:
```
module ActsAsFerret
  module ClassMethods
    def paginate_search(query, options = {})
      options, page, per_page = wp_parse_options!(options)
      pager = WillPaginate::Collection.new(page, per_page, nil)
      options.merge!(:offset =&gt; pager.offset, :limit =&gt; per_page)
      result = result = find_by_contents(query, options)
      result = result.delete_if { |result|
      !result.component_instance.viewable?(user) &amp;&amp; (result.parent.respond_to?('password') &amp;&amp; result.parent.component_instance.password(session, user))
    }
#change the total_hits to something like results.length
      returning WillPaginate::Collection.new(page, per_page, result.length) do |pager|
        pager.replace result
      end
    end
  end
end
```
Wait, now that I think about that, it’s quite absurd as the ferret query is automatically limited anyway :) I suppose the only way to really do it is to return all the results, filter them, then paginate from there. This shouldn’t be too expensive since the sites are quite small.

Thanks heaps :)
jags November 14, 2007

hello all
i need some help.
i was earlier using paginating_find and moved on to will_paginate now

i my view i am showing the total count of the ferret search results.

Search for “book” Produced Results

i have @products.size ————which failed
i tried @products.total_hits ——failed again
i made the result variable in AAF module as instance var and tried
@results.total_hits———-also failed

can anybody help me with this pls.
Jags

Everything from above worked for me
thanks a ton ……..thanks
Brendon November 14, 2007

Hi everyone, after a bit of working, I’ve come up with a solution to my problem, which will hopefully help those who want to do what I was doing (filtering search results after the search query has been run, then paginating them). Here’s the modified code:
```
module ActsAsFerret
  module ClassMethods
    def paginate_search(query, options = {}, &amp;block)
      options, page, per_page = wp_parse_options!(options)
      options.merge!(:limit =&gt; :all)
      results = find_by_contents(query, options)
      results.delete_if { |result| yield(result) }
      returning WillPaginate::Collection.new(page, per_page, results.length) do |pager|
        pager.replace results[pager.offset..pager.offset + pager.per_page - 1]
      end
    end
  end
end
```
Firstly, can someone explain the use of result = result = find_by… ? It seems a bit redundant to define the variable twice?

Anyway, basically how this works is we still parse the options etc… but instead of making find_by_contents get a page and an offset, we just get them all by defining :limit => :all. Then we run delete_if and yeild back to the calling method to provide us with a true false value, then we end up with a filtered set of results. We then manually build the result set by picking the right records from the array of results.

The calling method looks like:
```
    @results = Page.paginate_search("+domain_hash:\"#{domain_hash}\" AND #{@query}", :lazy =&gt; [:name], :page =&gt; params[:page], :per_page =&gt; 5, :models =&gt; :all){ |result|
      !result.component_instance.viewable?(user) || (result.parent.respond_to?('password') &amp;&amp; result.parent.component_instance.password(session, user))
    }
```
Hope this is of some use to somebody. :)

Cheers,

Brendon
jags November 16, 2007

hi brandon

can you pls check on my comment above.

how can i get the total_hits i.e the total no of results found for the search so that it can be displayed on the rhtml

pls help
thanks
jags
Johnny December 6, 2007

Hey man – thanks for a great piece of code. Works like a charm.
Cheers, JR
Frederico Araujo December 7, 2007

I wrote a basic way to get will_paginate to work with ferret
http://www.frederico-araujo.com/2007/12/7/search-with-ferret-using-will_paginate
Brad Carson December 7, 2007

Awesome! Works great! Thank you, Brandon!
Raecoo December 11, 2007

it’s works in my apps is fine.
Nice,Thank you,Brandon
Mike D December 26, 2007

Anyone know how to do this pagination when you’re doing ferret searches from >1 model? Can’t seem to get it to work! Thanks!
aiwilliams January 2, 2008

Is any of this necessary for the latest acts_as_ferret? find_by_contents’ second argument, the options, can include :page and :per_page, and the SearchResults returned from it has the appropriate extensions to work with the will_paginate view method.
Brandon January 2, 2008

aiwilliams,

Thanks for pointing that out. I haven’t tried it, but looking at the code, you appear to be correct.

Behrang March 12, 2008

There has been a change in the latest version of WillPaginate here is the code that is working for me:



module ActsAsFerret

  module ClassMethods

    def paginate_search(query, options = {})

      page, per_page, total = wp_parse_options(options)

      pager = WillPaginate::Collection.new(page, per_page, total)

      options.merge!(:offset => pager.offset, :limit => per_page)

      result = find_by_contents(query, options)

      returning WillPaginate::Collection.new(page, per_page, result.total_hits) do |pager|

        pager.replace result

      end

    end

  end

end

Cheers

Adam March 27, 2008

Hi – great work Brandon.

I want to search across about 10 models on one search page.

I saw a post by Jens: http://www.ruby-forum.com/topic/126739#565176 to use the :multi option with find_with_ferret and changed the lib file accordingly:

module ActsAsFerret

  module ClassMethods

    def paginate_search(query, options = {})

      page, per_page, total = wp_parse_options(options)

      pager = WillPaginate::Collection.new(page, per_page, total)

      options.merge!(:offset => pager.offset, :limit => per_page)

      result = find_with_ferret(query, options)

      returning WillPaginate::Collection.new(page, per_page, result.total_hits) do |pager|

        pager.replace result

      end

    end

  end

end

And this from my search controller…. but it only searches Model1:



@results = Model1.paginate_search @query,

           :page => @page, :per_page => @per_page,

           :multi => [Model2, Model3, Model4]

Adam March 28, 2008

Just confirming that aiwilliams was correct… Also, I noticed the syntax has changed for those of you who are using the latest version of AAF. Here is a sample to search across models and paginate (remember to use :store_class_name => true):
```
@results = ActsAsFerret::find(params[:q], [ModelA, ModelB, ModelC], {:page =&gt; params[:page], :per_page =&gt; 25})
```
izit April 19, 2008

Hi,

I am having some issues with this lib.

in my controller I use:

@mutations = Mutation.paginate_search(“test”,{:page=>1})

if I use @mutations in a render_partial in my view:

<%= render :partial => @mutations %>

I get the following error message:

ActiveRecord::DangerousAttributeError in Mutations#list

Showing mutations/mutation.rhtml where line #4 raised:

hash is defined by ActiveRecord

I have debugged this down to a problem that ActiveRecord does not see @mutations as an array.

If I do Mutations.find_with_ferret or Mutations.paginate I do not get the error. It only happens with function in the lib provided here. I use lates AAF (trunk) and willpaginate (2.2.1)
Santiago May 9, 2008

I put the code in lib/ferret.rb, then require ‘lib/ferret’ in environment.rb, after doing that I got the error below.

Removing the file back from lib solves the problem. What am I doing wrong? :(

Loading development environment (Rails 2.0.2)
/usr/lib/ruby/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:263:in `load_missing_constant’:NameError: uninitialized constant Ferret::Analysis::Analyzer
Avishai June 13, 2008

Awesome! I dropped the code into lib/acts_as_ferret_extension.rb and required it in environment.rb ~~- works very nicely, thanks :~~)
Joakim June 17, 2008

Actually, you don’t need any special code to do this.

At least the latest stable versions of acts_as_ferret (I’m using Rails 2.1) is fully compatible with will_paginate. Just use:

@products = Product.find_with_ferret(params[:q], {:page => params[:page], :per_page => 50}, find_options)
(where find_options is a hash with parameters you’d normally send to ActiveRecord::Base.find*, like :order, :conditions, :include, etc.)

Then just use <%= will_paginate @products %> like you normally would.

This is actually documented in the API docs for acts_as_ferret…
Alderete September 30, 2008

As Joakim writes, the latest versions of acts_as_ferret and will_paginate seem to work together for the basic pagination. But will_paginate includes another view helper method, e.g. <%= page_entries_info @people %>, to show the “showing xx-yy of zz” information, which is a nice supplement to the basic paging controls.

But, using this helper throws an exception “undefined method `total_entries’ for #”. Anyone have any ideas how to patch around this?
Alderete September 30, 2008

Let me answer my own question with what I finally figured out. Although I’m using the supposedly current 0.4.3 gem of acts_as_ferret, the documentation for acts_as_ferret asserts that total_hits is aliased to total_entries.

Maybe that’s true in a more recent version of acts_as_ferret, but for my application, I had to add the following:
```
module ActsAsFerret
  class SearchResults
    def total_entries
      @total_hits
    end
  end
end
```
(I added it to an initializer, but you can add it almost anywhere and just require it in.)
Fasal April 21, 2009

hiii Alderete
Thanks for your own question with self answer.I had the same kind of error and i put your code to enviornment.rb and that fixed the problem.Thanks once again for the answer.Have a great day.
Lucas Renan May 26, 2010

I tried to do a pagination with acts_as_ferret version 0.4.4 and will_paginate version 2.3.1.
I just did:

controller
```
@data = Model.find_with_ferret(params[“q”]+"~", :page => params[‘page’], :per_page => 10)
```
view
```
<%= will_paginate @data %>
```
and it seems work fine this way
David Lynch June 8, 2010

Hi Lucas,

I am using acts_as_version 0.4.4 and will_paginate 2.3.11, I have some what same code in my
controller
@ads = Ad.find_with_ferret(params[:q]+"~", :page => params[‘page’], :per_page => 10)

view
will_paginate @ads

But i only get one page of paginated ads can’t go second page just get nil results even do I know there is more that 10 result.

opensoul.org

acts_as_ferret will_paginate

49 Comments

Post a Comment

Popular Posts