David Yip [Mon, 13 Nov 2017 17:06:02 +0000 (11:06 -0600)]
Maintain case-insensitivity when merging multiple matchers (#213)
When given two regexps, Regexp.union preserves the options set (or not
set) on each regex; this meant that none of the multiline (m),
case-insensitivity (i), or extended syntax (x) options were set. Our
regexps are written expecting the m, i, and x options were set on all of
them, so we need to make sure that we preserve that behavior.
aschmitz [Sat, 11 Nov 2017 02:11:10 +0000 (20:11 -0600)]
Per-user reblog hiding implementation/fixes/tests
Note that this will only hide/show *future* reblogs by a user, and does
nothing to remove/add reblogs that are already in the timeline. I don't
think that's a particularly confusing behavior, and it's a lot easier
to implement (similar to mutes, I believe).
unarist [Fri, 27 Oct 2017 14:10:22 +0000 (23:10 +0900)]
Optimize FixReblogsInFeeds migration (#5538)
We have changed how we store reblogs in the redis for bigint IDs. This process is done by 1) scan all entries in users feed, and 2) re-store reblogs by 3 write commands.
However, this operation is really slow for large instances. e.g. 1hrs on friends.nico (w/ 50k users). So I have tried below tweaks.
* It checked non-reblogs by `entry[0] == entry[1]`, but this condition won't work because `entry[0]` is String while `entry[1]` is Float. Changing `entry[0].to_i == entry[1]` seems work.
-> about 4-20x faster (feed with less reblogs will be faster)
* Write operations can be batched by pipeline
-> about 6x faster
* Wrap operation by Lua script and execute by EVALSHA command. This really reduces packets between Ruby and Redis.
-> about 3x faster
I've taken Lua script way, though doing other optimizations may be enough.
David Yip [Tue, 24 Oct 2017 23:40:28 +0000 (18:40 -0500)]
Override Action View name inference in settings/keyword_mutes.
Glitch::KeywordMute's name is inferred as glitch_keyword_mutes, and in
templates this turns into e.g. settings/glitch/keyword_mutes. Going
along with this convention means a lot of file movement, though, and for
a UI that's as temporary and awkward as this one I think it's less
effort to slap a bunch of as: options everywhere.
We'll do the Right Thing when we build out the API and frontend UI.
David Yip [Tue, 24 Oct 2017 23:31:34 +0000 (18:31 -0500)]
Switch to Regexp.union for building the mute expression.
Also make the keyword-building methods private: they always probably
should have been private, but now I have encoded enough fun and games
into them that it now seems wrong for them to *not* be private.
David Yip [Tue, 24 Oct 2017 00:31:59 +0000 (19:31 -0500)]
Only cache the regex text, not the regex itself.
It is possible to cache a Regexp object, but I'm not sure what happens
if e.g. that object remains in cache across two different Ruby versions.
Caching a string seems to raise fewer questions.
Matthew Walsh [Mon, 16 Oct 2017 04:02:39 +0000 (21:02 -0700)]
Added a timeline for Direct statuses
* Lists all Direct statuses you've sent and received
* Displayed in Getting Started
* Streaming server support for direct TL
David Yip [Sun, 22 Oct 2017 05:24:32 +0000 (00:24 -0500)]
Don't add \b to whole-word keywords that don't start with word characters.
Ditto for ending with \b.
Consider muting the phrase "(hot take)". I stipulate it is reasonable
to enter this with the default "match whole word" behavior. Under the
old behavior, this would be encoded as
\b\(hot\ take\)\b
However, if \b is before the first character in the string and the first
character in the string is not a word character, then the match will
fail. Ditto for after. In our example, "(" is not a word character, so
this will not match statuses containing "(hot take)", and that's a very
surprising behavior.
To address this, we only add leading and trailing \b to keywords that
start or end with word characters.
David Yip [Sat, 21 Oct 2017 19:47:17 +0000 (14:47 -0500)]
Move KeywordMute into Glitch namespace.
There are two motivations for this:
1. It looks like we're going to add other features that require
server-side storage (e.g. user notes).
2. Namespacing glitchsoc modifications is a good idea anyway: even if we
do not end up doing (1), if upstream introduces a keyword-mute feature
that also uses a "KeywordMute" model, we can avoid some merge
conflicts this way and work on the more interesting task of
choosing which implementation to use.
David Yip [Mon, 16 Oct 2017 00:49:22 +0000 (19:49 -0500)]
Allow keywords to match either substrings or whole words.
Word-boundary matching only works as intended in English and languages
that use similar word-breaking characters; it doesn't work so well in
(say) Japanese, Chinese, or Thai. It's unacceptable to have a feature
that doesn't work as intended for some languages. (Moreso especially
considering that it's likely that the largest contingent on the Mastodon
bit of the fediverse speaks Japanese.)
There are rules specified in Unicode TR29[1] for word-breaking across
all languages supported by Unicode, but the rules deliberately do not
cover all cases. In fact, TR29 states
For example, reliable detection of word boundaries in languages such
as Thai, Lao, Chinese, or Japanese requires the use of dictionary
lookup, analogous to English hyphenation.
So we aren't going to be able to make word detection work with regexes
within Mastodon (or glitchsoc). However, for a first pass (even if it's
kind of punting) we can allow the user to choose whether they want word
or substring detection and warn about the limitations of this
implementation in, say, docs.
David Yip [Sun, 15 Oct 2017 08:17:33 +0000 (03:17 -0500)]
Set up /settings/keyword_mutes. #164.
This should eventually be accessible via the API and the web frontend,
but I find it easier to set up an editing interface using Rails
templates and the like. We can always take it out if it turns out we
don't need it.
David Yip [Sun, 15 Oct 2017 07:32:03 +0000 (02:32 -0500)]
Use more idiomatic string concatentation. #164.
The intent of the previous concatenation was to minimize object
allocations, which can end up being a slow killer. However, it turns
out that under MRI 2.4.x, the shove-strings-in-an-array-and-join method
is not only arguably more common but (in this particular case) actually
allocates *fewer* objects than the string concatenation.
Or, at least, that's what I gather by running this:
David Yip [Sun, 15 Oct 2017 01:45:14 +0000 (20:45 -0500)]
Make use of the regex attr_reader. #164.
It would also have been valid to get rid of the attr_reader, but I like
being able to reach inside KeywordMute::Matcher without resorting to
instance_variable_get tomfoolery.
David Yip [Sun, 15 Oct 2017 01:36:53 +0000 (20:36 -0500)]
Rework KeywordMute interface to use a matcher object; spec out matcher. #164.
A matcher object that builds a match from KeywordMute data and runs it
over text is, in my view, one of the easier ways to write examples for
this sort of thing.
David Yip [Mon, 9 Oct 2017 22:28:28 +0000 (17:28 -0500)]
Add KeywordMute model.
Gist of the proposed keyword mute implementation:
Keyword mutes are represented server-side as one keyword per record.
For each account, there exists a keyword regex that is generated as one
big alternation of all keywords. This regex is cached (in Redis, I
guess) so we can quickly get it when filtering in FeedManager.
David Yip [Thu, 19 Oct 2017 15:59:50 +0000 (10:59 -0500)]
Make the compose area optionally scrollable.
On desktop, the compose text box grows to accommodate the content. On
mobile, the text box does not grow to accommodate text context, but does
grow to accommodate images. It is possible in both cases to overflow
the available area, which makes accessing other UI elements (e.g.
visibility setttings) difficult.
This commit makes the compose area optionally scrollable, which allows
those UI elements to remain available even if they go off-screen.
David Yip [Wed, 18 Oct 2017 16:52:34 +0000 (11:52 -0500)]
Update stylesheet imports in glitch components.
Commit 6e5471947438fc5883e72b8184663564ffadee28 moved the Mastodon
variables and mixins deeper in the directory hierarchy; this commit
brings the glitch components in line with that change.
Technowix [Wed, 18 Oct 2017 11:51:30 +0000 (13:51 +0200)]
Revert #5438 for FR (#5450)
As said here https://github.com/tootsuite/mastodon/pull/5438 the point of shortening the timestamp is legit, and after some time of adaptation no mistakes can be mades.
cat aescling's git repositories / mastodon.git / log