Tip: Puppet augeas provider skips files

I was writing a Puppet class using the augeas provider and found a case where the manifest would work sometimes, but not other times, resulting in a silent error, revealed only by a debug message:

Skipping because no files were changed

I started with this, and it worked sometimes:

augeas {"automaster$name": context => "/files/etc/auto.master", changes => $changes, onlyif => "match map[. = '/$name'] size == 0" }

In some cases it did not work, saying "Skipping because no files were changed" even though debugging showed that changes were made. This version works all the time:

augeas {"automaster$name": context => "/files/etc/auto.master", incl => "/etc/auto.master", lens => "Automaster.lns", load_path => "/var/lib/puppet/lib/augeas/lenses", changes => $changes, onlyif => "match map[. = '/$name'] size == 0" }

I'm not yet sure why this happened, but it seems the difference is that in a manifest where augeas is used previously with load_path, that I need to use it here too.

Tip: Joining lists in Puppet

Tip of the day: joining lists in Puppet

node default { include test } class test { $list = ['foo','bar',undef,'4',undef,'six'] $joined = inline_template("<%= (list).reject{|x| x == :undef} %>") notify{"see $joined":} }

Another attempt...

A coworker of mine rekindled my thoughts about blogging. So, I thought I'd add a couple posts.

Posting Using Google CL

I posted this text using the blogger command line. http://code.google.com/p/googlecl/

TAI timestamp patch for rsyslog

Greetings,

I have a need to have rsyslog log TAI timestamps. For reference:

http://cr.yp.to/libtai/tai64.html

I have created a patch that appears to basically work.

Here's the rsyslog.conf to enable what I'm after:

$template tai,"%timegenerated:1:25:date-tai% <%syslogfacility%>%timegenerated% %msg%\n"
*.info;mail.none;authpriv.none;cron.none -/var/log/messages;tai

As a test, I perform some log activity with my tai template in action along with rfc3339, and compare the two templates. Note the use of *tai64nlocal* from D. Bernstein's daemontools package to validate the output. [url]http://cr.yp.to/daemontools/tai64nlocal.html[/url]

$template taiplus,"%timegenerated:1:25:date-tai% %timegenerated:1:25:date-rfc3339%%msg%\n"
*.info;mail.none;authpriv.none;cron.none -/var/log/messages;taiplus

[root@host ~] while true; do date | logger && tail -1 /var/log/messages | tai64nlocal ; sleep 1; done
2009-08-29 15:03:55.805317000 2009-08-29T15:03:55.80531 Sat Aug 29 15:03:55 CDT 2009
2009-08-29 15:03:56.819275000 2009-08-29T15:03:56.81927 Sat Aug 29 15:03:56 CDT 2009
2009-08-29 15:03:57.834073000 2009-08-29T15:03:57.83407 Sat Aug 29 15:03:57 CDT 2009

So, it appears that my TAI timestamp matches the RFC3339 timestamp.

Here's the patch. Feedback and/or review would be great. The tai.h and taia.h header files are in libtai, which I've compiled and installed in /usr/include/libtai/. [url]http://cr.yp.to/libtai.html[/url].

diff -ruN rsyslog-4.2.0.orig/runtime/datetime.c rsyslog-4.2.0/runtime/datetime.c
--- rsyslog-4.2.0.orig/runtime/datetime.c 2009-06-22 12:11:10.000000000 -0500
+++ rsyslog-4.2.0/runtime/datetime.c 2009-08-29 18:54:18.000000000 -0500
@@ -45,6 +45,12 @@
#include "stringbuf.h"
#include "errmsg.h"

+#include
+#include
+
+#define tai_unix(t,u) ((void) ((t)->x = 4611686018427387914ULL + (uint64) (u)))
+#define TIMESTAMP 25
+
/* static data */
DEFobjStaticHelpers
DEFobjCurrIf(errmsg)
@@ -670,6 +676,79 @@
}

/**
+ * Format a syslogTimestamp to TAI timestamp string.
+ */
+int formatTimestampTAI(struct syslogTime *ts, char* pBuf, size_t iLenBuf)
+{
+ int iRet = 0;
+ int i;
+ long offset;
+
+ struct taia taiatime;
+ struct tai taitime;
+ struct tm logtm = {0};
+ struct timeval logtimeval = {0};
+ time_t logtimet = 0;
+
+ char taiapack[TAIA_PACK];
+ char s[TIMESTAMP];
+
+ static char hex[16] = "0123456789abcdef";
+
+ assert(ts != NULL);
+ assert(pBuf != NULL);
+
+ if(iLenBuf < 25)
+ return(0); // we NEED at least 25 bytes
+
+ logtm.tm_year = ts->year - 1900;
+ logtm.tm_mon = ts->month - 1;
+ logtm.tm_mday = ts->day;
+ logtm.tm_hour = ts->hour;
+ logtm.tm_min = ts->minute;
+ logtm.tm_sec = ts->second;
+ logtm.tm_wday = 0;
+ logtm.tm_yday = 0;
+ logtm.tm_isdst = -1;
+ // adjust for 24 hour clock
+ if (logtm.tm_hour == 24)
+ {
+ logtm.tm_hour = 0;
+ }
+
+ offset = (int)ts->OffsetHour;
+ if (ts->OffsetMode == '+') offset *= -1;
+ logtm.tm_hour += offset;
+
+ offset = ts->OffsetMinute;
+ if (ts->OffsetMode == '+') offset *= -1;
+ logtm.tm_min += offset;
+
+ logtimet = mktime(&logtm);
+
+ logtimeval.tv_sec = logtimet;
+ tai_unix(&taiatime.sec,logtimeval.tv_sec);
+ taiatime.nano = 0;
+ taiatime.atto = 0;
+
+ if(ts->secfracPrecision > 0)
+ {
+ taiatime.nano = 1000*ts->secfrac;
+ }
+
+ // taiapack is string
+ taia_pack(taiapack,&taiatime);
+
+ s[0] = '@';
+ for (i = 0;i < 12;++i) {
+ s[i * 2 + 1] = hex[(taiapack[i] >> 4) & 15];
+ s[i * 2 + 2] = hex[taiapack[i] & 15];
+ }
+ iRet = snprintf(pBuf, iLenBuf, "%25.25s", s);
+ return(iRet);
+}
+
+/**
* Format a syslogTimestamp to a RFC3164 timestamp sring.
* The caller must provide the timestamp as well as a character
* buffer that will receive the resulting string. The function
@@ -738,6 +817,7 @@
pIf->formatTimestampSecFrac = formatTimestampSecFrac;
pIf->formatTimestamp3339 = formatTimestamp3339;
pIf->formatTimestamp3164 = formatTimestamp3164;
+ pIf->formatTimestampTAI = formatTimestampTAI;
finalize_it:
ENDobjQueryInterface(datetime)

diff -ruN rsyslog-4.2.0.orig/runtime/datetime.h rsyslog-4.2.0/runtime/datetime.h
--- rsyslog-4.2.0.orig/runtime/datetime.h 2009-06-22 12:11:10.000000000 -0500
+++ rsyslog-4.2.0/runtime/datetime.h 2009-08-29 18:39:23.000000000 -0500
@@ -43,6 +43,7 @@
int (*formatTimestamp3339)(struct syslogTime *ts, char* pBuf, size_t iLenBuf);
int (*formatTimestamp3164)(struct syslogTime *ts, char* pBuf, size_t iLenBuf);
int (*formatTimestampSecFrac)(struct syslogTime *ts, char* pBuf, size_t iLenBuf);
+ int (*formatTimestampTAI)(struct syslogTime *ts, char* pBuf, size_t iLenBuf);
ENDinterface(datetime)
#define datetimeCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */
/* interface changes:
diff -ruN rsyslog-4.2.0.orig/runtime/Makefile.in rsyslog-4.2.0/runtime/Makefile.in
--- rsyslog-4.2.0.orig/runtime/Makefile.in 2009-06-22 12:25:05.000000000 -0500
+++ rsyslog-4.2.0/runtime/Makefile.in 2009-08-29 18:39:23.000000000 -0500
@@ -222,7 +222,7 @@
INSTALL_PROGRAM = @INSTALL_PROGRAM@
INSTALL_SCRIPT = @INSTALL_SCRIPT@
INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@
-LDFLAGS = @LDFLAGS@
+LDFLAGS = @LDFLAGS@ -ltai
LIBDBI_CFLAGS = @LIBDBI_CFLAGS@
LIBDBI_LIBS = @LIBDBI_LIBS@
LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@
diff -ruN rsyslog-4.2.0.orig/runtime/msg.c rsyslog-4.2.0/runtime/msg.c
--- rsyslog-4.2.0.orig/runtime/msg.c 2009-06-22 12:11:10.000000000 -0500
+++ rsyslog-4.2.0/runtime/msg.c 2009-08-29 18:39:23.000000000 -0500
@@ -369,6 +369,8 @@
free(pThis->pszRcvdAt3164);
if(pThis->pszRcvdAt3339 != NULL)
free(pThis->pszRcvdAt3339);
+ if(pThis->pszRcvdAtTAI != NULL)
+ free(pThis->pszRcvdAtTAI);
if(pThis->pszRcvdAt_SecFrac != NULL)
free(pThis->pszRcvdAt_SecFrac);
if(pThis->pszRcvdAt_MySQL != NULL)
@@ -379,6 +381,8 @@
free(pThis->pszTIMESTAMP3164);
if(pThis->pszTIMESTAMP3339 != NULL)
free(pThis->pszTIMESTAMP3339);
+ if(pThis->pszTIMESTAMPTAI != NULL)
+ free(pThis->pszTIMESTAMPTAI);
if(pThis->pszTIMESTAMP_SecFrac != NULL)
free(pThis->pszTIMESTAMP_SecFrac);
if(pThis->pszTIMESTAMP_MySQL != NULL)
@@ -844,6 +848,16 @@
}
MsgUnlock(pM);
return(pM->pszTIMESTAMP3339);
+ case tplFmtTAIDate:
+ MsgLock(pM);
+ if(pM->pszTIMESTAMPTAI == NULL) {
+ if((pM->pszTIMESTAMPTAI= malloc(26)) == NULL) {
+ MsgUnlock(pM);
+ return ""; /* TODO: check this: can it cause a free() of constant memory?) */
+ }
+ datetime.formatTimestampTAI(&pM->tTIMESTAMP, pM->pszTIMESTAMPTAI, 26);
+ }
+ MsgUnlock(pM);
case tplFmtSecFrac:
MsgLock(pM);
if(pM->pszTIMESTAMP_SecFrac == NULL) {
@@ -922,6 +936,17 @@
}
MsgUnlock(pM);
return(pM->pszRcvdAt3339);
+ case tplFmtTAIDate:
+ MsgLock(pM);
+ if(pM->pszRcvdAtTAI == NULL) {
+ if((pM->pszRcvdAtTAI = malloc(26)) == NULL) {
+ MsgUnlock(pM);
+ return "";
+ }
+ datetime.formatTimestampTAI(&pM->tRcvdAt, pM->pszRcvdAtTAI, 26);
+ }
+ MsgUnlock(pM);
+ return(pM->pszRcvdAtTAI);
case tplFmtSecFrac:
MsgLock(pM);
if(pM->pszRcvdAt_SecFrac == NULL) {
diff -ruN rsyslog-4.2.0.orig/runtime/msg.h rsyslog-4.2.0/runtime/msg.h
--- rsyslog-4.2.0.orig/runtime/msg.h 2009-06-22 12:11:10.000000000 -0500
+++ rsyslog-4.2.0/runtime/msg.h 2009-08-29 18:39:23.000000000 -0500
@@ -110,12 +110,14 @@
struct syslogTime tRcvdAt;/* time the message entered this program */
char *pszRcvdAt3164; /* time as RFC3164 formatted string (always 15 charcters) */
char *pszRcvdAt3339; /* time as RFC3164 formatted string (32 charcters at most) */
+ char *pszRcvdAtTAI; /* time as TAI formatted string (25 charcters at most) */
char *pszRcvdAt_SecFrac;/* time just as fractional seconds (6 charcters) */
char *pszRcvdAt_MySQL; /* rcvdAt as MySQL formatted string (always 14 charcters) */
char *pszRcvdAt_PgSQL; /* rcvdAt as PgSQL formatted string (always 21 characters) */
struct syslogTime tTIMESTAMP;/* (parsed) value of the timestamp */
char *pszTIMESTAMP3164; /* TIMESTAMP as RFC3164 formatted string (always 15 charcters) */
char *pszTIMESTAMP3339; /* TIMESTAMP as RFC3339 formatted string (32 charcters at most) */
+ char *pszTIMESTAMPTAI; /* TIMESTAMP as TAI formatted string (25 charcters at most) */
char *pszTIMESTAMP_MySQL;/* TIMESTAMP as MySQL formatted string (always 14 charcters) */
char *pszTIMESTAMP_PgSQL;/* TIMESTAMP as PgSQL formatted string (always 21 characters) */
char *pszTIMESTAMP_SecFrac;/* TIMESTAMP fractional seconds (always 6 characters) */
diff -ruN rsyslog-4.2.0.orig/template.c rsyslog-4.2.0/template.c
--- rsyslog-4.2.0.orig/template.c 2009-06-22 12:11:10.000000000 -0500
+++ rsyslog-4.2.0/template.c 2009-08-29 18:39:23.000000000 -0500
@@ -496,6 +496,8 @@
pTpe->data.field.eDateFormat = tplFmtRFC3164Date;
} else if(!strcmp((char*)Buf, "date-rfc3339")) {
pTpe->data.field.eDateFormat = tplFmtRFC3339Date;
+ } else if(!strcmp((char*)Buf, "date-tai")) {
+ pTpe->data.field.eDateFormat = tplFmtTAIDate;
} else if(!strcmp((char*)Buf, "date-subseconds")) {
pTpe->data.field.eDateFormat = tplFmtSecFrac;
} else if(!strcmp((char*)Buf, "lowercase")) {
@@ -1132,6 +1134,9 @@
case tplFmtRFC3339Date:
dbgprintf("[Format as RFC3339-Date] ");
break;
+ case tplFmtTAIDate:
+ dbgprintf("[Format as TAI-Date] ");
+ break;
default:
dbgprintf("[INVALID eDateFormat %d] ", pTpe->data.field.eDateFormat);
}
diff -ruN rsyslog-4.2.0.orig/template.h rsyslog-4.2.0/template.h
--- rsyslog-4.2.0.orig/template.h 2009-06-22 12:11:10.000000000 -0500
+++ rsyslog-4.2.0/template.h 2009-08-29 18:39:23.000000000 -0500
@@ -48,7 +48,7 @@
enum EntryTypes { UNDEFINED = 0, CONSTANT = 1, FIELD = 2 };
enum tplFormatTypes { tplFmtDefault = 0, tplFmtMySQLDate = 1,
tplFmtRFC3164Date = 2, tplFmtRFC3339Date = 3, tplFmtPgSQLDate = 4,
- tplFmtSecFrac = 5};
+ tplFmtSecFrac = 5, tplFmtTAIDate = 6};
enum tplFormatCaseConvTypes { tplCaseConvNo = 0, tplCaseConvUpper = 1, tplCaseConvLower = 2 };

#include "msg.h"

GeekTool Weathermap

Let's try a code block on this blog, since I write code from time to time. I modified Blogger's stylesheet to do these nice block quotes with shaded backgrounds.

I use GeekTool on my Macbook Pro to display useful stuff on my desktop. One such thing I put together is a weather map of wherever I am, geographically. To do this, assume you have a network connection. Get your current IP. Determine the geographical location of that IP. Get the weather map for that location.

First, a bash shell script to get the weather map:

#!/bin/sh

# Modify path to add portage stuff, which is where "links" and "wget" etc. live.
PATH="/sw/bin:$PATH"

# Location for a saved weather map
MAP=/var/tmp/map.gif

# Determine our current IP address
IP=`links -dump http://checkip.dyndns.org/ | awk '{print $NF}'`

# Determine latitude and longitude
COOR=`wget -q -O - http://api.hostip.info/get_xml.php?ip=${IP} | awk '/coordinates/{print}' | tr -d '[[:alpha:]<>/: ]'`

# Split into individual components
LAT=`echo $COOR | awk -F, '{print $2}'`
LON=`echo $COOR | awk -F, '{print $1}'`

# Now get the weather map for the latitude and longitude
wget -q -O $MAP "http://radblast.wunderground.com/cgi-bin/radar/WUNIDS_composite?centerlat=${LAT}&centerlon=${LON}&radius=120&type=N0R&frame=0&num=5&delay=15&width=230&height=230&newmaps=1&r=1250167046"

# Log that this is complete
logger -p daemon.err -t "$0[$$]" "retrieved: map for ip:${IP} lat:${LAT} lon:${LON}"

Now, this is an example weather map that the above script downloaded:



The script saves the map. Then use GeekTool to display the map on the desktop.

First post

I'm experimenting with blogger, and blogs in general. This is an idle curiosity, not because I have anything in particular to say.

Let's just call this, "Further decreasing the the signal to noise ratio of the internet."