Archiving my Twitter, Facebook and other social network activity
I finally added some screenshots (and more troubleshooting tips) to my guide on how to post to #Mastodon from anything (including #RSS, #WordPress etc.) using #IFTTT
(Incidentally: I built a new recipe from scratch to make sure I wasn’t missing any steps and tried to publish it as a reusable applet, but it turns out IFTTT won’t let you publish applets that use webhooks.)
While messing around with #IndieWeb, I’ve decided I want to build a consolidated #PESOS gallery of my photos at Instagram, Twitter, Mastodon, Pixelfed etc. I’ve got current archives for all (including from a dead Pixelfed instance).
Specs:
* Gallery, individual, and tag galleries
* Include captions and descriptions (I can add descriptions to the old photos)
* Links to wherever it’s posted (may be several sites)
* Light page code. I’d rather things load quickly than act fancy.
1/3
I’ve made a prototype in #WordPress so far, and it looks nice, and I already know how to automate new posts using IFTTT. But it feels heavy. And of course it’s one more instance to increase my attack surface & maintenance.
I’m going to give #Eleventy a shot next. I’ve worked with it before, and I can optimize the templates however I want. The image plugin can generate different sizes, thumbnails, etc. The main drawback: I haven’t researched ways to automatically import to 11ty yet.
2/3
Long-term I’d like to add #POSSE (selectively) to make it easier to cross-post. I had a set of 3 photos yesterday that I wanted to post to Flickr, Pixelfed, my blog, and my gemlog, and while it was mostly copying and pasting, it was also time-consuming.
Some of the stuff @stringerblog and @dansup have been talking about kind of overlap with this space.
3/4
Oh yeah, Flickr…the way I use it is different enough that I don’t want include my zillions of photos in this project. I might try a more full-featured gallery like Piwigo at some point. It’s less linear and more collection-based. But for photos I’ve cross-posted, I would like to link to it same as I would to Pixelfed or wherever, and it would be nice to include it in the outgoing posts once I get that up and running.
And I do like the idea of just federating the photos out directly!
4/5
OK, this got way more complicated and should’ve been a blog post.
So: building a personal #IndieWeb capable gallery of my sociallly-shared photos from various sources, keeping it light, pulling in new posts automatically (#PESOS) and eventually posting on my gallery and auto-posting it to Pixelfed/Twitter/Flickr/etc. (#POSSE) on a per-photo basis. Probably using #eleventy.
Anyone have any recommendations on tools that might be better than WP/11ty, or that would work well with this workflow?
Ooh, I could also generate a Gemini gallery! I remember @FiXato came up with some ideas on how to format one. I’d probably want to break it into categories or do it by tag or something
I wonder how much storage this will end up taking up. I probably won’t need to use S3 (finally started setting things up on DigitalOcean instead of AWS!) for this gallery unless I really want to CDN the images. By the time they’re scaled for whatever social network, it doesn’t take up any more space than by blog storage.
If I ever make my own Flickr mirror, though, that’s going to have to be a consideration!
I don’t mind that IFTTT is introducing a Pro subscription for additional capabilities, and if there was something I wanted to do with those features, I’d go for it. But what I use it for isn’t worth $10/month.
The problem is that they’re limiting the free edition to just THREE applets.
Not cool.
https://ifttt.com/explore/introducing_ifttt_pro
I guess I need to look through what I’m actually using, clear out what I’m not, see what can be replaced and what I can consolidate. The name-your-own-price deal for the first year might be worth it just to have a cushion.
But that’s eventually going to balloon out to $10/month again and I honestly don’t see myself using enough features on there for it to be worth it.
I’ll have to look for a nextcloud app or something
Honestly, I don’t mind paying for stuff. I donate to free software projects. I contribute to this site’s admin patreon. But price points matter, and IFTTT is following the typical (and annoying) pattern for formerly-free services where they squash the free tier down hard and aim the lowest paid tier at people who are going to use a lot more of it than I am.
IFTTT has redone their website. Let’s see…
1. On IFTTT, click on Get More.
2. Up near the top, there’s a “Make more Applets from scratch” bar.. Click on the “IfThisThenThat” button.
3. Click on “This” in “If +This Then That”
4. Search for RSS and choose New feed item.
5. Ender the URL to your feed and click on “Create trigger”
6. Click on “That” in “If [rss] Then +That”
7. Search for Webhooks and click on it.
8. Choose Make a Web Request
Pick up with the article again!
@natharari I’m also going to update the article now that IFTTT has kind of hidden the way to get started on it.
@natharari It varies.
Usually it’ll start slowly, maybe every few hours, and as it gets a sense of how often the feed updates, it will adjust.
I have some other recipes that pull *from* my Mastodon and Pixelfed feeds, and it’s usually within 10 minutes on my Mastodon posts, but within a few hours on Pixelfed.
While fine-tuning the iNaturalist import, I started auto-importing images to the blog from the image in the feed, using media_sideload_image(). Since Pixelfed has started including an embedded image in its feed, I did the same there.
And I finally bit the bullet and wrote a simple feed proxy to read my Mastodon Atom feeds and rearrange the elements that IFTTT doesn’t know to look for: images attached as enclosures, and the post URL for boosted posts. (It was picking up the author’s URL because they’re both link rel=”alternate” and instead of actually looking at the feed structure – like the fact that the author’s link was inside an <author> entity – it was just pulling the first one it found.)
So in theory, I should be able to let this run and it will automatically import not only my Instagram photos, but Pixelfed, Mastodon, and iNaturalist as well. Twitter’s a bit more complex because it doesn’t link directly to the photo URL, so I’ll have to retrieve the link and parse the HTML to find it. And multi-photo posts are still an issue on both Pixelfed (because the feed only includes the first image) and Instagram (because the importer doesn’t handle it quite right). But that’s still a big improvement for my usual use case, and a lot less manual adjusting I need to do!
I updated my article on posting from #IFTTT to #Mastodon again to clarify a few things, and to recommend botsin.space for accounts that will *only* include automatic posts.
I use this technique for several things:
– Link to every new post from an #RSS feed at @SpeedForce
– Post differently tagged links from #Pocket here & at SpeedForce
– Cross-post a very occasional #WordPress blog here
@ibagail @packbat It’s possible to post from IFTTT to Mastodon using their webhook feature. I wrote up a blog post on how to do it:
https://hyperborea.org/journal/2017/12/mastodon-ifttt/
Let me know if you have any questions!
I also like the idea of boosting a blog post and having replies federate as comments.
We’ve all been trained by FB to discuss on the network instead of at the source. That fragments the conversation, and while sometimes that’s a good thing, sometimes it helps to, well, centralize.
Everyone who’s asked me for help using IFTTT with Mastodon has asked *here*, not on the blog post, so they haven’t seen other people who had the same problem & how they fixed it.
I’ve automated a few more things: Splitting retweets/boosts/etc to a separate “repost” author, for instance, adding tags, and setting formats where there’s enough information.
PixelFed.Social is being archived now, though it’s got the same problem as Mastodon where the media link doesn’t appear in the section IFTTT knows about, so I still have to manually add the image later on.
The Instagram/Twitter merge is done.
I keep changing my mind about how I want to handle cases where I posted the same photo to several sites, but cropped or filtered it differently. For now I’m grouping by version, so if I post a square crop to Instagram (which goes to Facebook and Twitter) and a rectangular crop to PixelFed and Photog.Social, I’m merging the Instagram/Facebook/Twitter posts into one and the PixelFed/Photog.Social posts into another.
I pulled in the AltBrowser Twitter archive, fixing truncated items and timezone in the spreadsheet before importing it.
I still haven’t gotten around to writing that Google Plus Archive to WordPress XML converter, but I’ve started going through and manually importing just the original posts at Google Plus. Not the auto-links to my blogs, except in a few cases where there’s a comment thread I want to be able to find again. I’ve found a few photos that I haven’t posted anywhere else, and at least one post that I later cross-posted to K2R.
Lately I’ve been phasing out Facebook and Twitter, and moved my primary Mastodon presence from Mastodon.Social to Wandering.Shop. The last few months, most of my Facebook posts have been cross-posts from other services, and now that they’ve shut down all of those (except Instagram, of course, since they own it), I basically haven’t been there at all.
I’ve updated my article on posting to Mastodon through #IFTTT to clarify that ‘status=’ is a required part of the body, and took the opportunity to verify that IFTTT can handle additional parameters like spoiler_text=___ for CWs, visibility=___ to set unlisted, etc.
You can chain them like this:
spoiler_text=Link: <<<{{Title}}>>>&visibility=unlisted&status=<<<{{Url}}>>> <<<{{Excerpt}}>>>
I’ve set up this site to archive my third-party social networking posts on a site that I control and can easily search. For now I’m setting up the following networks to archive here using IFTTT:
I’m less concerned with keeping everything in its original form, and more concerned with being able to find it (and work my way back to comment threads), so I plan on removing/combining duplicates as I find them, cleaning up links, etc. but I don’t want to get too complicated with it.
Last night I tested linking Pocket to Mastodon via #IFTTT to automatically share links when I add a certain tag. (See details). Today I discovered something else about IFTTT.
One of the links I used had an old tag to share it to Buffer, and Pocket cheerfully picked up that tag & sent it out again.
That means IFTTT doesn’t know what tags are *new* on a Pocket bookmark. It only knows that the bookmark has been updated, and what the current tags are.
I’m kind of reminded of XKCD’s initial response to Google+
Ultimately, though, I think the separate instances are going to be both the biggest roadblock to adoption and the key feature that distinguishes Mastodon from Twitter.
Wait, is there a taboo against naming that other microblogging site? that other microblogging site. Should I call it “You-know-where” or “That which must not be named?”
(I will admit I was amused to see someone refer to it as “The Bad Place” yesterday.)
…and I just noticed the editing error in this toot. :facepalm:
I do wonder about permanence, though: How often do Mastodon instances shut down? How much notice?
Data Export seems to only cover follow/block/mute lists so far. I suppose I could hook up my atom feed to IFTTT or something.
Or I can just do what I do with “the bird site,” and if I really want to make sure I keep something I’ve written, copy it over to my blog.