• Join - It's Free

API - Project returning too much data

Started by Private User on Monday, March 24, 2014
Problem with this page?

Participants:

Related Projects:

Showing all 24 posts

Seems that this..
http://www.geni.com/api/project-272/profiles?fields=id,name

Should be limited to primarily the id & name of each profile. However, it gives me a lot more than that.
It gives: url, public first_name, last_name, is_alive, gender, birth date, birth location, death date, death location, burial

If I'm trying to read all the profiles in this project, which is over 5000, most of the traffic is information that I've not requested via the "fields". This is not only a lot of overhead and traffic for you, but for me as well.

I'll take a look. Thanks for find this.

Try again, please. :)

Scott - what does the "G" next to your name in blue mean ???

Geni staff

thanks - I never saw this before

Looks great Private User! Thanks :)

Private User, looks like this problem might be true on the user as well.
https://www.geni.com/api/user/followed-profiles?fields=id,name&...

Also, is there any way to just get the followed count?

... from the documentation, it implies that one should be able to only get the one field (total count).

Sorry for the delayed reply. I'll try to get a look at this sometime this week.

We don't return the count for that call, just an array of results. The size of the array is the count.

The documentation is in error. I'll fix that.

I was incorrect about the size of the array. We paginate the results so the array size isn't the total count.

... so COULD it just return the total count (if that is all that is asked for)?

The pagination is probably done to save processing time. Finding the total count require that all entries have to be extracted from the DB to count them. Besides: The number cannot be trusted anyhow because it might change while you are processing the result, so it would just be informative anyhow.

Private User, don't forget the prior issue from 2:25pm where it is not using fields.

Dan Cornett I think we could return the total count with each page. So you could get it from just the first page of results and passing in a per_page=1 parameter would minimize the number of results returned as well - speeding up the response time a bit.

What do you need the count for?

Private User Sorry, I missed that question. I'll try to take a look in the next couple of days.

re: total count (followed profiles). Sure, it can change at any time.

For at least one intended use that number would be saved (for the "logged-in" user) at the beginning of a bunch of 'reading/processing' of profiles -- which may or may not make any changes to the followed status -- then to read that number again afterwards and show the user the difference. Exact value is not critical; just to show that changes did (or did not) likely occurred.

Right, to follow on Dan's comment. I'm looking at using the count for HistoryLink, which has a feature that will follow profiles in the search path, such as automatically follow X generations of my ancestors. I'd like to be able to show the follow count before and after running the tool. So in that case, I don't need anything but the count.

As for the DB, I don't think the DB extracts the data if you just run a "select count(*) where profileid=xxxx" SQL query if their DB is based on SQL. Or as Scott mentioned, adding the count as a field in the page. NOSQL / Graph database may be different.

Private User I think I fixed the 2:25pm issue. Please verify when you get a chance.

Private User, Looks good! :)

Dan Cornett I just added code that will include a total_count field if it's known. Here's a link that will show a minimal example

https://www.geni.com/api/user/followed-profiles?limit=2&fields=id

You have to pass limit=2 because limit=1 just gives you a has containing the single record.

Jeff: I think that ought to work for you; I just tried it (running that example before/after an unfollow/follow and the numbers returned seemed to be as expected.

Thanks, Scott!

Showing all 24 posts

Create a free account or login to participate in this discussion