In my WebApi controller I have the following (pseudo) code that receives update

Question

In my WebApi controller I have the following (pseudo) code that receives update notifications from Instagrams real-time API:

[HttpPost]
public void Post(InstagramUpdate instagramUpdate)
{
    var subscriptionId = instagramUpdate.SubscriptionId;
    var lastUpdate = GetLastUpdate(subscriptionId);

    // To avoid breaking my Instagram request limit, do not fetch new images too often.
    if (lastUpdate.AddSeconds(5) < DateTime.UtcNow)
    {
        // More than 5 seconds ago since last update for this subscription. Get new images
        GetNewImagesFromInstagram(subscriptionId);
        UpdateLastUpdate(subscriptionId, DateTime.UtcNow);
    }
}
This won't work very well if I receive two update notifications for the same subscription almost simultaneously, since lastUpdate won't have been updated until after the first request has been processed. And even if I update lastUpdate directly, there is a "lag" because of the round trip to the database.

What would be the best way to tackle this problem? I'm thinking of using some kind of cache (that would be faster than the database roundtrip), but I'm not sure how. Is there some kind of best practices for these kind of things? I'm guessing it's a common problem: "receive notification, do something if something hasn't been done recently..."

Explanation / Answer

I have another suggestion for the mechanism: when an update arrives, mark a subscription as having to update, and wait for your delay before making the update. That is, there is no need for a cache, you only need to store some state for each subscription id as "having to update" or not.

I will use async because it makes the code pretty straightforward.

For example:

private struct Empty {}
private ConcurrentDictionary<int, Empty> subscriptionsToUpdate = new ConcurrentDictionary<int, Empty>();

[HttpPost]
public async Task PostAsync(InstagramUpdate instagramUpdate)
{
var subscriptionId = instagramUpdate.SubscriptionId;

if (subscriptionsToUpdate.TryAdd(subscriptionId, default(Empty)) {
await Task.Delay(TimeSpan.FromSeconds(5));

if (subscriptionsToUpdate.TryRemove(subscriptionId)) {
GetNewImagesFromInstagram(subscriptionId);
UpdateLastUpdate(subscriptionId, DateTime.UtcNow);
}
}
}
Note that the implementation here only considers requests that are handled by a single Web API process. (E.g. if you have multiple servers, that code would potentially make one request, every five seconds, for each server that you have. If you want to avoid that, you would have to use some kind of shared store instead of a ConcurrentDictionary. (I would recommend something like a key value store, like Redis or memcached.))

Related Questions

Navigate

• Browse (All)
• Browse I
• Subjects

Integrity-first tutoring: explanations and feedback only — we do not complete graded work. Learn more.