pull down to refresh
Thank you for appreciating the challenge and giving me ideas!!
decaying algorithm based on last change to engagement.
Oh, my idea was just based on time (fetch older items less often), but basing it on the last change to engagement immediately made a lot more sense! My idea turned out to be a proxy metric for this, haha
Or maybe I can get @k00b to merge something, since I'm not dealing with a mega-corporation's API like it sounds in your case 👀
Migrations, while possible, are hell if you get it wrong at first.
I think in my case, in the worst case, I could just fetch every single item from Stacker News, starting from #1 all the way to #1545472. It might take a day or two, though.
(For this website, I fetched the feed for every territory. Bios and saloons aren't part of a territory, so I didn't fetch them or their replies.)
Oh, my idea was just based on time (fetch older items less often), but basing it on the last change to engagement immediately made a lot more sense! My idea turned out to be a proxy metric for this, haha
That is exactly where I started off. Then I ran into updates for old contentious posts becoming hot again coming along too slowly. (Posts were the main item.)
Or maybe I can get @k00b to merge something, since I'm not dealing with a mega-corporation's API like it sounds in your case 👀
That is by far the best outcome. It takes away the need of priority rotation and guessing.
Back then, I was dealing with Facebook on the other end. The tool was based on Workspace which was a (now discontinued) clone of Facebook's code for businesses to use as intranet. A neat idea, but it didn't make it.
Even though I was writing the tool for a huge and prestigious customer, we couldn't convince them to assign resources to add such a feed.
Anyway, that's my bit in adding to your plea for @koob to help the man out. :)
I (and my AI minions) was tinkering with a fun visualizer for SN but limiting it to the current daily Saloon:
- parent comments become tables and users in that thread are seated at the table, sorta replaying the thread
- popular tables (threads) get effects and stuff
- Music links show up as a jukebox or something
- image links and promos turn into posters on the walls or flyers
- watch it live, rewind the current saloon, or replay past Saloons
Ooof. I remember the last time I had to write code to handle that for a different system. It was brutal.
I don't remember all the details, but had to keep indexes weighing last engagements (likes/dislikes, comments and shares) and time passed since last check. Then introduce a decaying algorithm based on last change to engagement velocity.
Some types of engagement were more indicative of the need to check again as hot topics would often trigger waves of engagement.
And that all fed into a rotation of posts to revisit.
In that case, I had to do that dance as we were working with API limitations to the number of calls a tool could make. But it was a community with 90-100k users producing, on average, between 4 and 5 signals each per day.
Long story short: Good luck. And choose your data structures wisely from the get go. Migrations, while possible, are hell if you get it wrong at first.