These are the ramblings of Matthijs Kooijman, concerning the software he hacks on, hobbies he has and occasionally his personal life.
Most content on this site is licensed under the WTFPL, version 2 (details).
Questions? Praise? Blame? Feel free to contact me.
My old blog (pre-2006) is also still available.
See also my Mastodon page.
Sun | Mon | Tue | Wed | Thu | Fri | Sat |
---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 |
8 | 9 | 10 | 11 | 12 | 13 | 14 |
15 | 16 | 17 | 18 | 19 | 20 | 21 |
22 | 23 | 24 | 25 | 26 | 27 | 28 |
29 | 30 | 31 |
(...), Arduino, AVR, BaRef, Blosxom, Book, Busy, C++, Charity, Debian, Electronics, Examination, Firefox, Flash, Framework, FreeBSD, Gnome, Hardware, Inter-Actief, IRC, JTAG, LARP, Layout, Linux, Madness, Mail, Math, MS-1013, Mutt, Nerd, Notebook, Optimization, Personal, Plugins, Protocol, QEMU, Random, Rant, Repair, S270, Sailing, Samba, Sanquin, Script, Sleep, Software, SSH, Study, Supermicro, Symbols, Tika, Travel, Trivia, USB, Windows, Work, X201, Xanthe, XBee
On my server, I use LVM for managing partitions. I have one big "data" partition that is stored on an HDD, but for a bit more speed, I have an LVM cache volume linked to it, so commonly used data is cached on an SSD for faster read access.
Today, I wanted to resize the data volume:
# lvresize -L 300G tika/data
Unable to resize logical volumes of cache type.
Bummer. Googling for the error message showed me some helpful posts here and here that told me you have to remove the cache from the data volume, resize the data volume and then set up the cache again.
For this, they used lvconvert --uncache
, which detaches and deletes
the cache volume or cache pool completely, so you then have to recreate
the entire cache (and thus figure out how you created it in the first
place).
Trying to understand my own work from long ago, I looked through
documentation and found the lvconvert --splitcache
in
lvmcache(7), which detached a cache volume or cache pool,
but does not delete it. This means you can resize and just reattached
the cache again, which is a lot less work (and less error prone).
For an example, here is how the relevant volumes look:
# lvs -a
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data tika Cwi-aoC--- 300.00g [data-cache_cvol] [data_corig] 2.77 13.11 0.00
[data-cache_cvol] tika Cwi-aoC--- 20.00g
[data_corig] tika owi-aoC--- 300.00g
Here, data
is a "cache" type LV that ties together the big data_corig
LV
that contains the bulk data and small data-cache_cvol
that contains the
cached data.
After detaching the cache with --splitcache
, this changes to:
# lvs -a
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data tika -wi-ao---- 300.00g
data-cache tika -wi------- 20.00g
I think the previous data
cache LV was removed, data_corig
was renamed to
data
and data-cache_cvol
was renamed to data-cache
again.
Armed with this knowledge, here's how the ful resize works:
lvconvert --splitcache tika/data
lvresize -L 300G tika/data @hdd
lvconvert --type cache --cachevol tika/data-cache tika/data --cachemode writethrough
The last command might need some additional parameters depending on how you set
up the cache in the first place. You can view current cache parameters with
e.g. lvs -a -o +cache_mode,cache_settings,cache_policy
.
Note that all of this assumes using a cache volume an not a cache pool. I was originally using a cache pool setup, but it seems that a cache pool (which splits cache data and cache metadata into different volumes) is mostly useful if you want to split data and metadata over different PV's, which is not at all useful for me. So I switched to the cache volume approach, which needs fewer commands and volumes to set up.
I killed my cache pool setup with --uncache
before I found out about
--splitcache
, so I did not actually try --splitcache
with a cache pool, but
I think the procedure is actually pretty much identical as described above,
except that you need to replace --cachevol
with --cachepool
in the last
command.
For reference, here's what my volumes looked like when I was still using a cache pool:
# lvs -a
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
data tika Cwi-aoC--- 260.00g [data-cache] [data_corig] 99.99 19.39 0.00
[data-cache] tika Cwi---C--- 20.00g 99.99 19.39 0.00
[data-cache_cdata] tika Cwi-ao---- 20.00g
[data-cache_cmeta] tika ewi-ao---- 20.00m
[data_corig] tika owi-aoC--- 260.00g
This is a data
volume of type cache, that ties together the big data_corig
LV that contains the bulk data and a data-cache
LV of type cache-pool that
ties together the data-cache_cdata
LV with the actual cache data and
data-cache_cmeta
with the cache metadata.
A few months ago, I put up an old Atom-powered Supermicro server (SYS-5015A-PHF) again, to serve at De War to collect and display various sensor and energy data about our building.
The server turned out to have an annoying habit: every now and then it would start beeping (one continuous annoying beep), that would continue until the machine was rebooted. It happened sporadically, but kept coming back. When I used this machine before, it was located in a datacenter where nobody would care about a beep more or less (so maybe it has been beeping for years on end before I replaced the server), but now it was in a server cabinet inside our local Fablab, where there are plenty of people to become annoyed by a beeping server...
I eventually traced this back to faulty sensor readings and fixed this by disabling the faulty sensors completely in the server's IPMI unit, which will hopefully prevent the annoying beep. In this post, I'll share my steps, in case anyone needs to do the same.
When sorting out some stuff today I came across an "Ecobutton". When you attach it through USB to your computer and press the button, your computer goes to sleep (at least that is the intention).
The idea is that it makes things more sustainable because you can more easily put your computer to sleep when you walk away from it. As this tweakers poster (Dutch) eloquently argues, having more plastic and electronics produced in China, shipped to Europe and sold here for €18 or so probably does not have a net positive effect on the environment or your wallet, but well, given this button found its way to me, I might as well see if I can make it do something useful.
I had previously started a project to make a "Next" button for spotify that you could carry around and would (wirelessly - with an ESP8266 inside) skip to the next song using the Spotify API whenever you pressed it. I had a basic prototype working, but then the project got stalled on figuring out an enclosure and finding sufficiently low-power addressable RGB LEDs (documentation about this is lacking, so I resorted to testing two dozen different types of LEDs and creating a website to collect specs and test results for adressable LEDs, which then ended up with the big collection of other Yak-shaving projects waiting for this magical moment where I suddenly have a lot of free time).
In any case, it seemed interesting to see if this Ecobutton could be used as poor-man's spotify next button. Not super useful, but at least now I can keep the button around knowing I can actually use it for something in the future. I also produced some useful (and not readily available) documentation about remapping keys with hwdb in the process, so it was at least not a complete waste of time... Anyway, into the technical details...
Recently, a customer asked me te have a look at an external hard disk he was using with his Macbook. It would show up a file listing just fine, but when trying to open actual files, it would start failing. Of course there was no backup, but the files were very precious...
This started out as a small question, but ended up in an adventure that spanned a few days and took me deep into the ddrescue recovery tool, through the HFS+ filesystem and past USB power port control. I learned a lot, discovered some interesting things and produced a pile of scripts that might be helpful to others. Since the journey seems interesting as well as the end result, I will describe the steps I took here, "ter leering ende vermaeck".
Every now and then I work on some complex C++ code (mostly stuff running on Arduino nowadays) so I can write up some code in a nice, consise and abstracted manner. This almost always involves classes, constructors and templates, which serve their purpose in the abstraction, but once you actually call them, the compiler should optimize all of them away as much as possible.
This usually works nicely, but there was one thing that kept bugging me. No matter how simple your constructors are, initializing using constructors always results in some code running at runtime.
In contrast, when you initialize normal integer variable, or a struct variable using aggregate initialization, the copmiler can completely do the initialization at compiletime. e.g. this code:
struct Foo {uint8_t a; bool b; uint16_t c};
Foo x = {0x12, false, 0x3456};
Would result in four bytes (0x12, 0x00, 0x34, 0x56, assuming no padding and big-endian) in the data section of the resulting object file. This data section is loaded into memory using a simple loop, which is about as efficient as things get.
Now, if I write the above code using a constructor:
struct Foo {
uint8_t a; bool b; uint16_t c;};
Foo(uint8_t a, bool b, uint16_t c) : a(a), b(b), c(c) {}
};
Foo x = Foo(0x12, false, 0x3456);
This will result in those four bytes being allocated in the bss section (which is zero-initialized), with the constructor code being executed at startup. The actual call to the constructor is inlined of course, but this still means there is code that loads every byte into a register, loads the address in a register, and stores the byte to memory (assuming an 8-bit architecture, other architectures will do more bytes at at time).
This doesn't matter much if it's just a few bytes, but for larger objects, or multiple small objects, having the loading code intermixed with the data like this easily requires 3 to 4 times as much code as having it loaded from the data section. I don't think CPU time will be much different (though first zeroing memory and then loading actual data is probably slower), but on embedded systems like Arduino, code size is often limited, so not having the compiler just resolve this at compiletime has always frustrated me.
Today I learned about a new feature in C++11: Constant initialization. This means that any global variables that are initialized to a constant expression, will be resolved at runtime and initialized before any (user) code (including constructors) starts to actually run.
A constant expression is essentially an expression that the compiler can
guarantee can be evaluated at compiletime. They are required for e.g array
sizes and non-type template parameters. Originally, constant expressions
included just simple (arithmetic) expressions, but since C++11 you can
also use functions and even constructors as part of a constant
expression. For this, you mark a function using the constexpr
keyword,
which essentially means that if all parameters to the function are
compiletime constants, the result of the function will also be
(additionally, there are some limitations on what a constexpr function
can do).
So essentially, this means that if you add constexpr
to all
constructors and functions involved in the initialization of a variable,
the compiler will evaluate them all at compiletime.
(On a related note - I'm not sure why the compiler doesn't deduce
constexpr
automatically. If it can verify if it's allowed to use
constexpr
, why not add it? Might be too resource-intensive perhaps?)
Note that constant initialization does not mean the variable has to be
declared const
(e.g. immutable) - it's just that the initial value
has to be a constant expression (which are really different concepts -
it's perfectly possible for a const
variable to have a non-constant
expression as its value. This means that the value is set by normal
constructor calls or whatnot at runtime, possibly with side-effects,
without allowing any further changes to the value after that).
Anyway, so much for the introduction of this post, which turned out longer than I planned :-). I learned about this feature from this great post by Andrzej Krzemieński. He also writes that it is not really possible to enforce that a variable is constant-initialized:
It is difficult to assert that the initialization of globals really took place at compile-time. You can inspect the binary, but it only gives you the guarantee for this binary and is not a guarantee for the program, in case you target for multiple platforms, or use various compilation modes (like debug and retail). The compiler may not help you with that. There is no way (no syntax) to require a verification by the compiler that a given global is const-initialized.
If you accidentially forget constexpr on one function involved, or some other requirement is not fulfilled, the compiler will happily fall back to less efficient runtime initialization instead of notifying you so you can fix this.
This smelled like a challenge, so I set out to investigate if I could
figure out some way to implement this anyway. I thought of using a
non-type template argument (which are required to be constant
expressions by C++), but those only allow a limited set of types to be
passed. I tried using builtin_constant_p
, a non-standard gcc
construct, but that doesn't seem to recognize class-typed constant
expressions.
static_assert
It seems that using the (also introduced in C++11) static_assert
statement is a reasonable (though not perfect) option. The first
argument to static_assert
is a boolean that must be a constant
expression. So, if we pass it an expression that is not a constant
expression, it triggers an error. For testing, I'm using this code:
class Foo {
public:
constexpr Foo(int x) { }
Foo(long x) { }
};
Foo a = Foo(1);
Foo b = Foo(1L);
We define a Foo
class, which has two constructors: one accepts an
int
and is constexpr
and one accepts a long
and is not
constexpr
. Above, this means that a
will be const-initialized, while
b
is not.
To use static_assert
, we cannot just pass a
or b
as the condition,
since the condition must return a bool type. Using the comma operator
helps here (the comma accepts two operands, evaluates both and then
discards the first to return the second):
static_assert((a, true), "a not const-initialized"); // OK
static_assert((b, true), "b not const-initialized"); // OK :-(
However, this doesn't quite work, neither of these result in an error. I
was actually surprised here - I would have expected them both to fail,
since neither a
nor b
is a constant expression. In any case, this
doesn't work. What we can do, is simply copy the initializer used for
both into the static_assert
:
static_assert((Foo(1), true), "a not const-initialized"); // OK
static_assert((Foo(1L), true), "b not const-initialized"); // Error
This works as expected: The int
version is ok, the long
version
throws an error. It doesn't trigger the assertion, but
recent gcc versions show the line with the error, so it's good enough:
test.cpp:14:1: error: non-constant condition for static assertion
static_assert((Foo(1L), true), "b not const-initialized"); // Error
^
test.cpp:14:1: error: call to non-constexpr function ‘Foo::Foo(long int)’
This isn't very pretty though - the comma operator doesn't make it very clear what we're doing here. Better is to use a simple inline function, to effectively do the same:
template <typename T>
constexpr bool ensure_const_init(T t) { return true; }
static_assert(ensure_const_init(Foo(1)), "a not const-initialized"); // OK
static_assert(ensure_const_init(Foo(1L)), "b not const-initialized"); // Error
This achieves the same result, but looks nicer (though the
ensure_const_init
function does not actually enforce anything, it's
the context in which it's used, but that's a matter of documentation).
Note that I'm not sure if this will actually catch all cases, I'm not
entirely sure if the stuff involved with passing an expression to
static_assert
(optionally through the ensure_const_init
function) is
exactly the same stuff that's involved with initializing a variable with
that expression (e.g. similar to the copy constructor issue below).
The function itself isn't perfect either - It doesn't handle (const) (rvalue) references so I believe it might not work in all cases, so that might need some fixing.
Also, having to duplicate the initializer in the assert statement is a big downside - If I now change the variable initializer, but forget to update the assert statement, all bets are off...
constexpr
constantAs Andrzej pointed out in his post, you can mark variables with
constexpr
, which requires them to be constant initialized. However,
this also makes the variable const
, meaning it cannot be changed after
initialization, which we do not want. However, we can still leverage this
using a two-step initialization:
constexpr Foo c_init = Foo(1); // OK
Foo c = c_init;
constexpr Foo d_init = Foo(1L); // Error
Foo d = d_init;
This isn't very pretty either, but at least the initializer is only defined once. This does introduce an extra copy of the object. With the default (implicit) copy constructor this copy will be optimized out and constant initialization still happens as expected, so no problem there.
However, with user-defined copy constructors, things are diffrent:
class Foo2 {
public:
constexpr Foo2(int x) { }
Foo2(long x) { }
Foo2(const Foo2&) { }
};
constexpr Foo2 e_init = Foo2(1); // OK
Foo2 e = e_init; // Not constant initialized but no error!
Here, a user-defined copy constructor is present that is not declared
with constexpr
. This results in e
being not constant-initialized,
even though e_init
is (this is actually slighly weird - I would expect
the initialization syntax I used to also call the copy constructor when
initializing e_init
, but perhaps that one is optimized out by gcc in
an even earlier stage).
We can user our earlier ensure_const_init
function here:
constexpr Foo f_init = Foo(1);
Foo f = f_init;
static_assert(ensure_const_init(f_init), "f not const-initialized"); // OK
constexpr Foo2 g_init = Foo2(1);
Foo2 g = g_init;
static_assert(ensure_const_init(g_init), "g not const-initialized"); // Error
This code is actually a bit silly - of course f_init
and g_init
are
const-initialized, they are declared constexpr
. I initially tried this
separate init variable approach before I realized I could (need to,
actually) add constexpr
to the init variables. However, this silly
code does catch our problem with the copy constructor. This is just a
side effect of the fact that the copy constructor is called when the
init variables are passed to the ensure_const_init
function.
One variant of the above would be to simply define two objects: the one you want, and an identical constexpr version:
Foo h = Foo(1);
constexpr Foo h_const = Foo(1);
It should be reasonable to assume that if h_const
can be
const-initialized, and h
uses the same constructor and arguments, that
h
will be const-initialized as well (though again, no real guarantee).
This assumes that the h_const
object, being unused, will be optimized
away. Since it is constexpr, we can also be sure that there are no
constructor side effects that will linger, so at worst this wastes a bit
of memory if the compiler does not optimize it.
Again, this requires duplication of the constructor arguments, which can be error-prone.
There's two significant problems left:
None of these approaches actually guarantee that
const-initialization happens. It seems they catch the most common
problem: Having a non-constexpr
function or constructor involved,
but inside the C++ minefield that is (copy) constructors, implicit
conversions, half a dozen of initialization methods, etc., I'm
pretty confident that there are other caveats we're missing here.
None of these approaches are very pretty. Ideally, you'd just write something like:
constinit Foo f = Foo(1);
or, slightly worse:
Foo f = constinit(Foo(1));
Implementing the second syntax seems to be impossible using a function -
function parameters cannot be used in a constant expression (they could
be non-const). You can't mark parameters as constexpr
either.
I considered to use a preprocessor macro to implement this. A macro
can easily take care of duplicating the initialization value (and since
we're enforcing constant initialization, there's no side effects to
worry about). It's tricky, though, since you can't just put a
static_assert
statement, or additional constexpr
variable
declaration inside a variable initialization. I considered using a
C++11 lambda expression for that, but those can only contain a
single return statement and nothing else (unless they return void
) and
cannot be declared constexpr
...
Perhaps a macro that completely generates the variable declaration and
initialization could work, but still a single macro that generates
multiple statement is messy (and the usual do {...} while(0)
approach
doesn't work in global scope. It's also not very nice...
Any other suggestions?
Update 2020-11-06: It seems that C++20 has introduced a new
keyword, constinit
to do exactly this: Require that at variable is
constant-initialized, without also making it const
like constexpr
does. See https://en.cppreference.com/w/cpp/language/constinit
Or: Forcing Linux to use the USB HID driver for a non-standards-compliant USB keyboard.
For an interactive art installation by the Spullenmannen, a friend asked me to have a look at an old paint mixing terminal that he wanted to use. The terminal is essentially a small computer, in a nice industrial-looking sealed casing, with a (touch?) screen, keyboard and touchpad. It was by "Lacour" and I think has been used to control paint mixing machines.
They had already gotten Linux running on the system, but could not get the keyboard to work and asked me if I could have a look.
The keyboard did work in the BIOS and grub (which also uses the BIOS), so we know it worked. Also, the BIOS seemed pretty standard, so it was unlikely that it used some very standard protocol or driver and I guessed that this was a matter of telling Linux which driver to use and/or where to find the device.
Inside the machine, it seemed the keyboard and touchpad were separate devices, controlled by some off-the-shelf microcontroller chip (probably with some custom software inside). These devices were connected to the main motherboard using a standard 10-pin expansion header intended for external USB ports, so it seemed likely that these devices were USB ports.
I was previously running an ancient Windows XP install under Virtualbox for the occasional time I needed Windows for something. However, since Debian Stretch, virtualbox is no longer supplied, due to security policy problems, I've been experimenting with QEMU, KVM and virt-manager. Migrating my existing VirtualBox XP installation to virt-manager didn't work (it simply wouldn't boot), and I do not have any spare Windows keys lying around, but I do have a Windows 7 installed alongside my Linux on a different partition, so I decided to see if I could get that to boot inside QEMU/KVM.
An obvious problem is the huge change in hardware between the real and virtual environment, but apparently recent Windows versions don't really mind this in terms of drivers, but the activation process could be a problem, especially when booting both virtually and natively. So far I have not seen any complications with either drivers or activation, not even after switching to virtio drivers (see below). I am using an OEM (preactivated?) version of Windows, so that might help in this area.
Update: When booting Windows in the VM a few weeks later, it started bugging me that my Windows was not genuine, and it seems no longer activated. Clicking the "resolve now" link gives a broken webpage, and going through system properties suggests to contact Lenovo (my laptop provider) to resolve this (or buy a new license). I'm not yet sure if this is really problematic, though. This happened shortly after replacing my hard disk, though I'm not sure if that's actually related.
Rebooting into Windows natively shows it is activated (again or still), but booting it virtually directly after that still shows as not activated...
Booting the installation was actually quite painless: I just used the
wizard inside virt-manager, entered /dev/sda
(my primary hard disk) as
the storage device, pressed start, selected to boot Windows in my
bootloader and it booted Windows just fine.
Booting is not really fast, but once it runs, things are just a bit sluggish but acceptable.
One caveat is that this adds the entire disk, not just the Windows partition. This also means the normal bootloader (grub in my case) will be used inside the VM, which will happily boot the normal default operating system. Protip: Don't boot your Linux installation inside a VM inside that same Linux installation, both instances will end up fighting in your filesystem. Thanks for fsck, which seems to have fixed the resulting garbage so far...
To prevent this, make sure to actually select your Windows installation in the bootloader. See below for a more permanent solution.
In some Arduino / C++ project, I was using a custom assert()
macro, that, if
the assertion would fail show an error message, along with the current
filename and line number. The filename was automatically retrieved using
the __FILE__
macro. However, this macro returns a full path, while we
only had little room to show it, so we wanted to show the filename only.
Until now, we've been storing the full filename, and when an assert was triggered we would use the strrchr function to chop off all but the last part of the filename (commonly called the "basename") and display only that. This works just fine, but it is a waste of flash memory, storing all these (mostly identical) paths. Additionally, when an assertion fails, you want to get a message out ASAP, since who knows what state your program is in.
Neither of these is really a showstopper for this particular project,
but I suspected there would be some way to use C++ constexpr
functions
and templates to force the compiler to handle this at compiletime, and
only store the basename instead of the full path. This week, I took up
the challenge and made something that works, though it is not completely
pretty yet.
Working out where the path ends and the basename starts is fairly easy
using something like strrchr
. Of course, that's a runtime version, but
it is easy to do a constexpr
version by implementing it recursively,
which allows the compiler to evaluate these functions at compiletime.
For example, here are constexpr
versions of strrchrnul()
,
basename()
and strlen()
:
/**
* Return the last occurence of c in the given string, or a pointer to
* the trailing '\0' if the character does not occur. This should behave
* just like the regular strrchrnul function.
*/
constexpr const char *static_strrchrnul(const char *s, char c) {
/* C++14 version
if (*s == '\0')
return s;
const char *rest = static_strrchr(s + 1, c);
if (*rest == '\0' && *s == c)
return s;
return rest;
*/
// Note that we cannot implement this while returning nullptr when the
// char is not found, since looking at (possibly offsetted) pointer
// values is not allowed in constexpr (not even to check for
// null/non-null).
return *s == '\0'
? s
: (*static_strrchrnul(s + 1, c) == '\0' && *s == c)
? s
: static_strrchrnul(s + 1, c);
}
/**
* Return one past the last separator in the given path, or the start of
* the path if it contains no separator.
* Unlike the regular basename, this does not handle trailing separators
* specially (so it returns an empty string if the path ends in a
* separator).
*/
constexpr const char *static_basename(const char *path) {
return (*static_strrchrnul(path, '/') != '\0'
? static_strrchrnul(path, '/') + 1
: path
);
}
/** Return the length of the given string */
constexpr size_t static_strlen(const char *str) {
return *str == '\0' ? 0 : static_strlen(str + 1) + 1;
}
So, to get the basename of the current filename, you can now write:
constexpr const char *b = static_basename(__FILE__);
However, that just gives us a pointer halfway into the full string literal. In practice, this means the full string literal will be included in the link, even though only a part of it is referenced, which voids the space savings we're hoping for (confirmed on avr-gcc 4.9.2, but I do not expect newer compiler version to be smarter about this, since the linker is involved).
To solve that, we need to create a new char
array variable that
contains just the part of the string that we really need. As happens
more often when I look into complex C++ problems, I came across a post
by Andrzej Krzemieński, which shows a technique to concatenate two
constexpr
strings at compiletime (his blog has a lot of great posts on
similar advanced C++ topics, a recommended read!). For this, he has a
similar problem: He needs to define a new variable that contains the
concatenation of two constexpr strings.
For this, he uses some smart tricks using parameter packs (variadic
template arguments), which allows to declare an array and set its
initial value using pointer
references (e.g. char foo[] = {ptr[0], ptr[1], ...}
). One caveat is
that the length of the resulting string is part of its type, so must be
specified using a template argument. In the concatenation case, this can
be easily derived from the types of the strings to concat, so that gives
nice and clean code.
In my case, the length of the resulting string depends on the contents of the string itself, which is more tricky. There is no way (that I'm aware of, suggestions are welcome!) to deduce a template variable based on the value of an non-template argument automatically. What you can do, is use constexpr functions to calculate the length of the resulting string, and explicitly pass that length as a template argument. Since you also need to pass the contents of the new string as a normal argument (since template parameters cannot be arbitrary pointer-to-strings, only addresses of variables with external linkage), this introduces a bit of duplication.
Applied to this example, this would look like this:
constexpr char *basename_ptr = static_basename(__FILE__);
constexpr auto basename = array_string<static_strlen(basename_ptr)>(basename_ptr); \
This uses the static_string
library published along with the above
blogpost. For this example to work, you will need some changes to the
static_string class (to make it accept regular char*
as well), see
this pull request for the version I used.
The resulting basename variable is an array_string
object, which
contains just a char
array containing the resulting string. You can
use array indexing on it directly to access variables, implicitly
convert to const char*
or explicitly convert using basename.c_str()
.
So, this solves my requirement pretty neatly (saving a lot of flash
space!). It would be even nicer if I did not need to repeat the
basename_ptr
above, or could move the duplication into a helper class
or function, but that does not seem to be possible.
I recently upgraded my systems to Debian Stretch, which caused GnuPG to stop working within Mutt. I'm not exactly sure what was wrong, but I discovered that GnuPG version 2 changed quite some things and relies more heavily on the gpg-agent, and I discovered that recent SSH version can forward unix domain socket instead of just TCP sockets, which allows forwarding a gpg-agent connection over SSH.
Until now, I had my GPG private keys stored on my server, Tika, where my Mutt mail client also runs. However, storing private keys, even with a passphrase, on permanentely connected multi-user system never felt quite right. So this seemed like a good opportunity to set up proper forwarding for my gpg agent, and keep my private keys confined to my laptop.
I already had some small scripts in place to easily connect to my server through SSH, attach to the remote tmux session (or start it), set up some port forwards (in particular a reverse port forward for SSH so my mail client and IRC client could open links in my browser), and quickly reconnect when the connection fails. However, once annoyance was that when the connection fails, the server might not immediately notice, so reconnecting usually left me with failed port forwards (since the remote listening port was still taken by the old session). This seemed like a good occasion to fix that as wel.
The end result is a reasonably complex script, that is probably worth
sharing here. The script can be found in my scripts git repository.
On the server, it calls an attach
script, but that's not much more
than attaching to tmux, or starting a new session with some windows if
no session is running yet.
The script is reasonably well-commented, including an introduction on what it can do, so I will not repeat that here.
For the GPG forwarding, I based upon this blogpost. There, they
suggest configuring an extra-socket
in gpg-agent.conf
, but I've
found that gpg-agent already created an extra socket (whose path I could
query with gpgconf --list-dirs
), so I didn't use that extra-socket
configuration line. They also talk about setting StreamLocalBindUnlink
to clean up a lingering socket when creating a new one, but that is
already handled by my script instead.
Furthermore, to prevent a gpg-agent from being autostarted by gnupg
serverside (in case the forwarding fails, or when I would connect
without this script, etc.), I added no-autostart
to
~/.gnupg/gpg.conf
. I'm not running systemd user session on my server,
but if you are you might need to disable or mask some ssh-agent sockets
and/or services to prevent systemd from creating sockets for ssh-agent
and starting it on-demand.
My next step is to let gpg-agent also be my ssh-agent (or perhaps just use plain ssh-agent) to enforce confirming each SSH authentication request. I'm currently using gnome-keyring / seahorse as my SSH agent, but that just silently approves everything, which doesn't really feel secure.
For a fair amount of years now, I've been using Mutt as my primary email client. It's a very nice text-based email client that is permanently running on my server (named drsnuggles). This allows me to connect to my server from anywhere, connect to the running Screen and always get exactly the same, highly customized, mail interface (some people will say that their webmail interfaces will allow for exactly the same, but in my experience webmail is always clumsy and slow compared to a decent, well-customized text-based client when processing a couple of hundreds of emails per day).
So I like my mutt / screen setup. However, there has been one particular issue that didn't work quite as efficient: attachments. Whenever I wanted to open an email attachment, I needed to save the attachment within mutt to some place that was shared through HTTP, make the file world-readable (mutt insists on not making your saved attachments world-readable), browse to some url on the local machine and open the attachment. Not quite efficient at all.
Yesterday evening I was finally fed up with all this stuff and decided to hack up a fix. It took a bit of fiddling to get it right (and I had nearly started to spend the day coding a patch for mutt when the folks in #mutt pointed out an easier, albeit less elegant "solution"), but it works now: I can select an attachment in mutt, press "x" and it gets opened on my laptop. Coolness.
Just in case anyone else is interested in this solution, I'll document how it works. The big picture is as follows: When I press "x", a mutt macro is invoked that copies the attachment to my laptop and opens the attachment there. There's a bunch of different steps involved here, which I'll detail below.