Welcome to TiddlyWiki created by Jeremy Ruston, Copyright © 2007 UnaMesa Association
[I believe this was fixed sometime before Solaris 10u4 (08/07).]
There is a [[known issue|http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6421427]] with Sun's new ZFS filesystem which affects some oldish ~IDE-based machines. The symptoms are very poor fs performance, with the machine becoming almost unusable when under load: keystrokes taking seconds to echo and so on. This affects several machines I use for test and development inclusing Netra X1s, Ultra 5s and 10s, and probably other machines of that era. This issue affects at least Solaris 10 u2 (6/06) and u3 (11/06).
A rather horrible workaround is to wrap the disks in SVM metadevices, and then built the zfs pools on top of those devices. This seems to insulate ZFS just enough from the underlying devices that it does not try and do whatever thing it's doing which is hurting performance so badly.
To construct a zfs on a mirror of two slices, {{{c0t0d0s7}}} and {{{c0t2d0s7}}}, you can do the following:
{{{
# metainit d14 1 1 c0t0d0s7
d14: Concat/Stripe is setup
# metainit d24 1 1 c0t2d0s7
d24: Concat/Stripe is setup
# zpool create export mirror /dev/md/dsk/d14 /dev/md/dsk/d24
warning: device in use checking failed: No such device
warning: device in use checking failed: No such device
# zpool list
NAME SIZE USED AVAIL CAP HEALTH ALTROOT
export 93.5G 58.5K 93.5G 0% ONLINE -
# zpool status export
pool: export
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
export ONLINE 0 0 0
mirror ONLINE 0 0 0
/dev/md/dsk/d14 ONLINE 0 0 0
/dev/md/dsk/d24 ONLINE 0 0 0
errors: No known data errors
}}}
A filesystem configured this way on a Netra X1 seems to have equal or better performance to a UFS filesystem: it's about as good as UFS on a single disk and better than UFS on a mirror. A ZFS filesystem built on the underlying slices has catastrophically worse performance than an equivalent UFS filesystem.
As the SVM devices are simply encapsulating disk slices the date written to disk is the same as would be if the pool was built on the slices. You can export and then reimport the ZFS pool, and it will then appear on the underlying devices. This will be useful when the fix for the issue appears: this is rumoured to be in u3.
See [[A ZFS workaround for old IDE machines in Solaris 10]]
A very confusing thing which can happen on Solaris is that the system can be clearly running out of memory, while {{{swap -l}}} says that there is plenty of free swap. Naïvely it seems as if {{{swap}}} is lying. But in fact it's not, it is just very confusing.
The crucial thing is that there is a difference between //allocated// and //reserved// memory:
* allocated memory is memory which some process has asked for and has been actually touched. So the memory pages physically exist, and may need to be swapped out at some point.
* reserved memory is memory which a process has laid claim to but not yet actually touched. These memory pages do not yet physically exist but may do at some future time.
There are two approaches to reserved memory you could take.
* Fast and loose, otherwise known as "overcommit": if something reserves memory just say yes. Do not check that you might need to make good on your promise. If you ever do need to make good on it, you just hope that at that point there is enough memory/swap. If there isn't then you have a problem which is not easy to solve – what process should you kill to free up memory? Linux does this (possibly for some values of "Linux").
* Cautious. If something reserves memory check that you do have enough swap to make good on that reservation: don't actually allocate memory/swap, but note it is reserved. This means you will never have a bad experience when asked to make good on the reservation (but you might have to fail reservations). Solaris does this.
So: {{{swap -l}}} only shows you allocated memory, and is thus horribly confusing if you have a substantial amount of reserved-but-not-allocated memory. {{{swap -d}}} //does// show you reserved memory and should show that there is no available swap when a machine is running out of memory. It's particularly annoying that {{{swap -l}}} claims that anything not allocated is free, which is just not the case (so, actually, it is lying).
There is a [[good document|http://developers.sun.com/solaris/articles/virtual_memory_arrays.html]] by Sun which describes this problem, and an approach to providing overcmmmit in a principled way on Solaris.
These instructions are specifically for Solaris 10, but they should be similar for any reasonably recent versions. The aim is to be able to do a standard desktop login over VNC.
!Requirements & assumptions
* An Xvnc server. I'll assume you're using the one that comes on the SFW disk, and lives at {{{/opt/sfw/bin/Xvnc}}} with its related programs ({{{vncpasswd}}}) in the same directory. Others should be similar.
* The machine is headless, and you want the Xvnc server to be the {{{:0}}} display. If it has a screen you'll want to make Xvnc be a higher-numbered display.
!Procedure
# Make a copy of the {{{Xservers}}} config file: <html>
<pre># mkdir -p /etc/dt
# (cd /usr/dt && tar cf - config/Xservers) | (cd /etc/dt && tar xfp -)
</pre></html>
# Edit {{{/etc/dt/config/Xservers}}}, and:
## assuming the machine is headless (see above) comment out the line beginning {{{:0}}} (with preceeding whitespace, I don't know why.
## Add a line: <html>
<pre>:0 Local local_uid@none nobody /opt/sfw/bin/Xvnc :0 -depth 24 -geometry 1280x1024 -rfbauth /etc/dt/vncpasswd -desktop par -fp tcp/localhost:7100 -localhost</pre></html> This means:
*** run as {{{nobody}}};
*** use the {{{/opt/sfw/bin/Xvnc}}} server as display 0;
*** colour depth is 24 bit (truecolour, 8 bits per pixel);
*** display will be 1280x1024;
*** VNC password file is {{{/etc/dt/vncpasswd}}};
*** desktop name is {{{par}}} – change to your taste, the machine I took this from is called par;
*** use a local font server on port 7100;
*** only allow connections from {{{localhost}}}.
# make a VNC password file: <html>
<pre># vncpasswd /etc/dt/vncpasswd
Password:
Verify:
# chgrp nobody /etc/dt/vncpasswd
# chmod g+r /etc/dt/vncpasswd</pre></html>
# Start {{{dtlogin}}}. The way you do this has changed in recent Solaris 10 releases as things are now looked after by SMF. So in a recent (post 6/06?) release you do:<html>
<pre>
# svcadm restart cde-login
</pre>
while in an older 10 release and in earlier Solaris versions you do:
<pre>
# /etc/init.d/dtlogin start
</pre></html>
Now you should be able to connect to the VNC server and get a normal Solaris GUI login (without sound etc). Since it will only allow connections from the local host, you'll probably need to tunnel things over ssh:
{{{
$ ssh -L5900:localhost:5900 par
$ vncviewer localhost:0
}}}
or something similar with ~PuTTY etc.
/***
|''Name:''|CryptoFunctionsPlugin|
|''Description:''|Support for cryptographic functions|
***/
//{{{
if(!version.extensions.CryptoFunctionsPlugin) {
version.extensions.CryptoFunctionsPlugin = {installed:true};
//--
//-- Crypto functions and associated conversion routines
//--
// Crypto "namespace"
function Crypto() {}
// Convert a string to an array of big-endian 32-bit words
Crypto.strToBe32s = function(str)
{
var be = Array();
var len = Math.floor(str.length/4);
var i, j;
for(i=0, j=0; i<len; i++, j+=4) {
be[i] = ((str.charCodeAt(j)&0xff) << 24)|((str.charCodeAt(j+1)&0xff) << 16)|((str.charCodeAt(j+2)&0xff) << 8)|(str.charCodeAt(j+3)&0xff);
}
while (j<str.length) {
be[j>>2] |= (str.charCodeAt(j)&0xff)<<(24-(j*8)%32);
j++;
}
return be;
};
// Convert an array of big-endian 32-bit words to a string
Crypto.be32sToStr = function(be)
{
var str = "";
for(var i=0;i<be.length*32;i+=8)
str += String.fromCharCode((be[i>>5]>>>(24-i%32)) & 0xff);
return str;
};
// Convert an array of big-endian 32-bit words to a hex string
Crypto.be32sToHex = function(be)
{
var hex = "0123456789ABCDEF";
var str = "";
for(var i=0;i<be.length*4;i++)
str += hex.charAt((be[i>>2]>>((3-i%4)*8+4))&0xF) + hex.charAt((be[i>>2]>>((3-i%4)*8))&0xF);
return str;
};
// Return, in hex, the SHA-1 hash of a string
Crypto.hexSha1Str = function(str)
{
return Crypto.be32sToHex(Crypto.sha1Str(str));
};
// Return the SHA-1 hash of a string
Crypto.sha1Str = function(str)
{
return Crypto.sha1(Crypto.strToBe32s(str),str.length);
};
// Calculate the SHA-1 hash of an array of blen bytes of big-endian 32-bit words
Crypto.sha1 = function(x,blen)
{
// Add 32-bit integers, wrapping at 32 bits
add32 = function(a,b)
{
var lsw = (a&0xFFFF)+(b&0xFFFF);
var msw = (a>>16)+(b>>16)+(lsw>>16);
return (msw<<16)|(lsw&0xFFFF);
};
// Add five 32-bit integers, wrapping at 32 bits
add32x5 = function(a,b,c,d,e)
{
var lsw = (a&0xFFFF)+(b&0xFFFF)+(c&0xFFFF)+(d&0xFFFF)+(e&0xFFFF);
var msw = (a>>16)+(b>>16)+(c>>16)+(d>>16)+(e>>16)+(lsw>>16);
return (msw<<16)|(lsw&0xFFFF);
};
// Bitwise rotate left a 32-bit integer by 1 bit
rol32 = function(n)
{
return (n>>>31)|(n<<1);
};
var len = blen*8;
// Append padding so length in bits is 448 mod 512
x[len>>5] |= 0x80 << (24-len%32);
// Append length
x[((len+64>>9)<<4)+15] = len;
var w = Array(80);
var k1 = 0x5A827999;
var k2 = 0x6ED9EBA1;
var k3 = 0x8F1BBCDC;
var k4 = 0xCA62C1D6;
var h0 = 0x67452301;
var h1 = 0xEFCDAB89;
var h2 = 0x98BADCFE;
var h3 = 0x10325476;
var h4 = 0xC3D2E1F0;
for(var i=0;i<x.length;i+=16) {
var j,t;
var a = h0;
var b = h1;
var c = h2;
var d = h3;
var e = h4;
for(j = 0;j<16;j++) {
w[j] = x[i+j];
t = add32x5(e,(a>>>27)|(a<<5),d^(b&(c^d)),w[j],k1);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=16;j<20;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),d^(b&(c^d)),w[j],k1);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=20;j<40;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),b^c^d,w[j],k2);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=40;j<60;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),(b&c)|(d&(b|c)),w[j],k3);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
for(j=60;j<80;j++) {
w[j] = rol32(w[j-3]^w[j-8]^w[j-14]^w[j-16]);
t = add32x5(e,(a>>>27)|(a<<5),b^c^d,w[j],k4);
e=d; d=c; c=(b>>>2)|(b<<30); b=a; a = t;
}
h0 = add32(h0,a);
h1 = add32(h1,b);
h2 = add32(h2,c);
h3 = add32(h3,d);
h4 = add32(h4,e);
}
return Array(h0,h1,h2,h3,h4);
};
}
//}}}
/***
|''Name:''|DeprecatedFunctionsPlugin|
|''Description:''|Support for deprecated functions removed from core|
***/
//{{{
if(!version.extensions.DeprecatedFunctionsPlugin) {
version.extensions.DeprecatedFunctionsPlugin = {installed:true};
//--
//-- Deprecated code
//--
// @Deprecated: Use createElementAndWikify and this.termRegExp instead
config.formatterHelpers.charFormatHelper = function(w)
{
w.subWikify(createTiddlyElement(w.output,this.element),this.terminator);
};
// @Deprecated: Use enclosedTextHelper and this.lookaheadRegExp instead
config.formatterHelpers.monospacedByLineHelper = function(w)
{
var lookaheadRegExp = new RegExp(this.lookahead,"mg");
lookaheadRegExp.lastIndex = w.matchStart;
var lookaheadMatch = lookaheadRegExp.exec(w.source);
if(lookaheadMatch && lookaheadMatch.index == w.matchStart) {
var text = lookaheadMatch[1];
if(config.browser.isIE)
text = text.replace(/\n/g,"\r");
createTiddlyElement(w.output,"pre",null,null,text);
w.nextMatch = lookaheadRegExp.lastIndex;
}
};
// @Deprecated: Use <br> or <br /> instead of <<br>>
config.macros.br = {};
config.macros.br.handler = function(place)
{
createTiddlyElement(place,"br");
};
// Find an entry in an array. Returns the array index or null
// @Deprecated: Use indexOf instead
Array.prototype.find = function(item)
{
var i = this.indexOf(item);
return i == -1 ? null : i;
};
// Load a tiddler from an HTML DIV. The caller should make sure to later call Tiddler.changed()
// @Deprecated: Use store.getLoader().internalizeTiddler instead
Tiddler.prototype.loadFromDiv = function(divRef,title)
{
return store.getLoader().internalizeTiddler(store,this,title,divRef);
};
// Format the text for storage in an HTML DIV
// @Deprecated Use store.getSaver().externalizeTiddler instead.
Tiddler.prototype.saveToDiv = function()
{
return store.getSaver().externalizeTiddler(store,this);
};
// @Deprecated: Use store.allTiddlersAsHtml() instead
function allTiddlersAsHtml()
{
return store.allTiddlersAsHtml();
}
// @Deprecated: Use refreshPageTemplate instead
function applyPageTemplate(title)
{
refreshPageTemplate(title);
}
// @Deprecated: Use story.displayTiddlers instead
function displayTiddlers(srcElement,titles,template,unused1,unused2,animate,unused3)
{
story.displayTiddlers(srcElement,titles,template,animate);
}
// @Deprecated: Use story.displayTiddler instead
function displayTiddler(srcElement,title,template,unused1,unused2,animate,unused3)
{
story.displayTiddler(srcElement,title,template,animate);
}
// @Deprecated: Use functions on right hand side directly instead
var createTiddlerPopup = Popup.create;
var scrollToTiddlerPopup = Popup.show;
var hideTiddlerPopup = Popup.remove;
// @Deprecated: Use right hand side directly instead
var regexpBackSlashEn = new RegExp("\\\\n","mg");
var regexpBackSlash = new RegExp("\\\\","mg");
var regexpBackSlashEss = new RegExp("\\\\s","mg");
var regexpNewLine = new RegExp("\n","mg");
var regexpCarriageReturn = new RegExp("\r","mg");
}
//}}}
About half of all local network performance problems are duplex mismatches.
The two ends of an ethernet connection (typically the host and switch) need to agree speed and duplex. They do this by negotiation: each end lets the other know what capabilities it has, and they agree the best compromise.
If one end refuses to negotiate, the other end is duty bound to fall back to half-duplex.
This means that, if (for instance) a switch if forced to 100/full and negotiation is turned off, then a host connecting to it will fall back to 100/half and there will then be a duplex mismatch. This is a depressingly common scenario.
The solution to this is:
* either make sure the host has speed & duplex forced;
* or (much better) let the two ends negotiate - don't disable negotiation on the switch.
People often force speed & duplex out of a mistaken understanding that this is how to ensure you get (say) 100/full. Also there were, once upon a time, some devices which got the negotiation wrong.
!Tools for Solaris
* {{{ndd}}} can let you set and query parameters for an interface interactively. There are generally two sets of parameters for an interface: whether to negotiate at all, and what to offer. If you want to make sure you get a particular result then the answer is to leave negotiation on, but only to offer the result you want. This is better than turning negotiation off.
* If you have to force parameters (for instance because the network people insist on turning negotiation off on the switch, and you are not allowed to kill them), then you can set them in {{{/etc/system}}} and/or in the driver configuration file in {{{/kernel/drv}}}. {{{/etc/system}}} is better, but if you need to find where someone else has set them it's worth checking the driver configuration files as wel. You may also find people forcing things via init scripts.
* In Solaris 10, {{{kstat}}} provides a very good way of finding all the parameters and statistics a kernel module (such as the driver for a NIC) has. I suspect {{{ndd}}} is now just a wrapper around {{{kstat}}}.
!Examples for Solaris
Here's a machine with an {{{eri}}}:
{{{
# ndd /dev/eri \?
? (read only)
transceiver_inuse (read only)
link_status (read only)
link_speed (read only)
link_mode (read only)
ipg1 (read and write)
ipg2 (read and write)
use_int_xcvr (read and write)
pace_size (read and write)
adv_autoneg_cap (read and write)
adv_100T4_cap (read and write)
adv_100fdx_cap (read and write)
adv_100hdx_cap (read and write)
adv_10fdx_cap (read and write)
adv_10hdx_cap (read and write)
autoneg_cap (read only)
100T4_cap (read only)
100fdx_cap (read only)
100hdx_cap (read only)
10fdx_cap (read only)
10hdx_cap (read only)
lp_autoneg_cap (read only)
lp_100T4_cap (read only)
lp_100fdx_cap (read only)
lp_100hdx_cap (read only)
lp_10fdx_cap (read only)
lp_10hdx_cap (read only)
instance (read and write)
lance_mode (read and write)
ipg0 (read and write)
intr_blank_time (read and write)
intr_blank_packets (read and write)
}}}
We want to query instance 0:
{{{
# ndd -set /dev/eri instance 0
}}}
Does it have autonegotiation?
{{{
# ndd -get /dev/eri autoneg_cap
1
}}}
now ask it if it is advertising autonegotiation
{{{
# ndd -get /dev/eri adv_autoneg_cap
1
}}}
finally, is it advertising 100 full?
{{{
# ndd -get /dev/eri adv_100fdx_cap
1
}}}
/***
|Name|HoverMenuPlugin|
|Created by|SaqImtiaz|
|Location|http://lewcid.googlepages.com/lewcid.html#HoverMenuPlugin|
|Version|1.11|
|Requires|~TW2.x|
!Description:
Provides a hovering menu on the edge of the screen for commonly used commands, that scrolls with the page.
!Demo:
Observe the hovering menu on the right edge of the screen.
!Installation:
Copy the contents of this tiddler to your TW, tag with systemConfig, save and reload your TW.
To customize your HoverMenu, edit the HoverMenu shadow tiddler.
To customize whether the menu sticks to the right or left edge of the screen, and its start position, edit the HoverMenu configuration settings part of the code below. It's well documented, so don't be scared!
The menu has an id of hoverMenu, in case you want to style the buttons in it using css.
!Notes:
Since the default HoverMenu contains buttons for toggling the side bar and jumping to the top of the screen and to open tiddlers, the ToggleSideBarMacro, JumpMacro and the JumpToTopMacro are included in this tiddler, so you dont need to install them separately. Having them installed separately as well could lead to complications.
If you dont intend to use these three macros at all, feel free to remove those sections of code in this tiddler.
!To Do:
* rework code to allow multiple hovering menus in different positions, horizontal etc.
* incorporate code for keyboard shortcuts that correspond to the buttons in the hovermenu
!History:
*03-08-06, ver 1.11: fixed error with button tooltips
*27-07-06, ver 1.1 : added JumpMacro to hoverMenu
*23-07-06
!Code
***/
/***
start HoverMenu plugin code
***/
//{{{
config.hoverMenu={};
//}}}
/***
HoverMenu configuration settings
***/
//{{{
config.hoverMenu.settings={
align: 'right', //align menu to right or left side of screen, possible values are 'right' and 'left'
x: 1, // horizontal distance of menu from side of screen, increase to your liking.
y: 158 //vertical distance of menu from top of screen at start, increase or decrease to your liking
};
//}}}
//{{{
//continue HoverMenu plugin code
config.hoverMenu.handler=function()
{
var theMenu = createTiddlyElement(document.getElementById("contentWrapper"), "div","hoverMenu");
theMenu.setAttribute("refresh","content");
theMenu.setAttribute("tiddler","HoverMenu");
var menuContent = store.getTiddlerText("HoverMenu");
wikify(menuContent,theMenu);
var Xloc = this.settings.x;
Yloc =this.settings.y;
var ns = (navigator.appName.indexOf("Netscape") != -1);
function SetMenu(id)
{
var GetElements=document.getElementById?document.getElementById(id):document.all?document.all[id]:document.layers[id];
if(document.layers)GetElements.style=GetElements;
GetElements.sP=function(x,y){this.style[config.hoverMenu.settings.align]=x +"px";this.style.top=y +"px";};
GetElements.x = Xloc;
GetElements.y = findScrollY();
GetElements.y += Yloc;
return GetElements;
}
window.LoCate_XY=function()
{
var pY = findScrollY();
ftlObj.y += (pY + Yloc - ftlObj.y)/15;
ftlObj.sP(ftlObj.x, ftlObj.y);
setTimeout("LoCate_XY()", 10);
}
ftlObj = SetMenu("hoverMenu");
LoCate_XY();
};
window.old_lewcid_hovermenu_restart = restart;
restart = function()
{
window.old_lewcid_hovermenu_restart();
config.hoverMenu.handler();
};
setStylesheet(
"#hoverMenu .button, #hoverMenu .tiddlyLink {border:none; font-weight:bold; background:#18f; color:#FFF; padding:0 5px; float:right; margin-bottom:4px;}\n"+
"#hoverMenu .button:hover, #hoverMenu .tiddlyLink:hover {font-weight:bold; border:none; color:#fff; background:#000; padding:0 5px; float:right; margin-bottom:4px;}\n"+
"#hoverMenu .button {width:100%; text-align:center}"+
"#hoverMenu { position:absolute; width:7px;}\n"+
"\n","hoverMenuStyles");
config.macros.renameButton={};
config.macros.renameButton.handler = function(place,macroName,params,wikifier,paramString,tiddler)
{
if (place.lastChild.tagName!="BR")
{
place.lastChild.firstChild.data = params[0];
if (params[1]) {place.lastChild.title = params[1];}
}
};
config.shadowTiddlers["HoverMenu"]="<<top>>\n<<toggleSideBar>><<renameButton '>' >>\n<<jump j '' top>>\n<<saveChanges>><<renameButton s 'Save TiddlyWiki'>>\n<<newTiddler>><<renameButton n>>\n";
//}}}
//end HoverMenu plugin code
//Start ToggleSideBarMacro code
//{{{
config.macros.toggleSideBar={};
config.macros.toggleSideBar.settings={
styleHide : "#sidebar { display: none;}\n"+"#contentWrapper #displayArea { margin-right: 1em;}\n"+"",
styleShow : " ",
arrow1: "«",
arrow2: "»"
};
config.macros.toggleSideBar.handler=function (place,macroName,params,wikifier,paramString,tiddler)
{
var tooltip= params[1]||'toggle sidebar';
var mode = (params[2] && params[2]=="hide")? "hide":"show";
var arrow = (mode == "hide")? this.settings.arrow1:this.settings.arrow2;
var label= (params[0]&¶ms[0]!='.')?params[0]+" "+arrow:arrow;
var theBtn = createTiddlyButton(place,label,tooltip,this.onToggleSideBar,"button HideSideBarButton");
if (mode == "hide")
{
(document.getElementById("sidebar")).setAttribute("toggle","hide");
setStylesheet(this.settings.styleHide,"ToggleSideBarStyles");
}
};
config.macros.toggleSideBar.onToggleSideBar = function(){
var sidebar = document.getElementById("sidebar");
var settings = config.macros.toggleSideBar.settings;
if (sidebar.getAttribute("toggle")=='hide')
{
setStylesheet(settings.styleShow,"ToggleSideBarStyles");
sidebar.setAttribute("toggle","show");
this.firstChild.data= (this.firstChild.data).replace(settings.arrow1,settings.arrow2);
}
else
{
setStylesheet(settings.styleHide,"ToggleSideBarStyles");
sidebar.setAttribute("toggle","hide");
this.firstChild.data= (this.firstChild.data).replace(settings.arrow2,settings.arrow1);
}
return false;
}
setStylesheet(".HideSideBarButton .button {font-weight:bold; padding: 0 5px;}\n","ToggleSideBarButtonStyles");
//}}}
//end ToggleSideBarMacro code
//start JumpToTopMacro code
//{{{
config.macros.top={};
config.macros.top.handler=function(place,macroName)
{
createTiddlyButton(place,"^","jump to top",this.onclick);
}
config.macros.top.onclick=function()
{
window.scrollTo(0,0);
};
config.commands.top =
{
text:" ^ ",
tooltip:"jump to top"
};
config.commands.top.handler = function(event,src,title)
{
window.scrollTo(0,0);
}
//}}}
//end JumpToStartMacro code
//start JumpMacro code
//{{{
config.macros.jump= {};
config.macros.jump.handler = function (place,macroName,params,wikifier,paramString,tiddler)
{
var label = (params[0] && params[0]!=".")? params[0]: 'jump';
var tooltip = (params[1] && params[1]!=".")? params[1]: 'jump to an open tiddler';
var top = (params[2] && params[2]=='top') ? true: false;
var btn =createTiddlyButton(place,label,tooltip,this.onclick);
if (top==true)
btn.setAttribute("top","true")
}
config.macros.jump.onclick = function(e)
{
if (!e) var e = window.event;
var theTarget = resolveTarget(e);
var top = theTarget.getAttribute("top");
var popup = Popup.create(this);
if(popup)
{
if(top=="true")
{createTiddlyButton(createTiddlyElement(popup,"li"),'Top ↑','Top of TW',config.macros.jump.top);
createTiddlyElement(popup,"hr");}
story.forEachTiddler(function(title,element) {
createTiddlyLink(createTiddlyElement(popup,"li"),title,true);
});
}
Popup.show(popup,false);
e.cancelBubble = true;
if (e.stopPropagation) e.stopPropagation();
return false;
}
config.macros.jump.top = function()
{
window.scrollTo(0,0);
}
//}}}
//end JumpMacro code
//utility functions
//{{{
Popup.show = function(unused,slowly)
{
var curr = Popup.stack[Popup.stack.length-1];
var rootLeft = findPosX(curr.root);
var rootTop = findPosY(curr.root);
var rootHeight = curr.root.offsetHeight;
var popupLeft = rootLeft;
var popupTop = rootTop + rootHeight;
var popupWidth = curr.popup.offsetWidth;
var winWidth = findWindowWidth();
if (isChild(curr.root,'hoverMenu'))
var x = config.hoverMenu.settings.x;
else
var x = 0;
if(popupLeft + popupWidth+x > winWidth)
popupLeft = winWidth - popupWidth -x;
if (isChild(curr.root,'hoverMenu'))
{curr.popup.style.right = x + "px";}
else
curr.popup.style.left = popupLeft + "px";
curr.popup.style.top = popupTop + "px";
curr.popup.style.display = "block";
addClass(curr.root,"highlight");
if(config.options.chkAnimate)
anim.startAnimating(new Scroller(curr.popup,slowly));
else
window.scrollTo(0,ensureVisible(curr.popup));
}
window.isChild = function(e,parentId) {
while (e != null) {
var parent = document.getElementById(parentId);
if (parent == e) return true;
e = e.parentNode;
}
return false;
};
//}}}
Sun's patch tools – {{{patchadd}}} & {{{patchrm}}} – have notoriously poor performance. This is a serious problem for organisations which need to meet uptime requirements or to fit changes into constrained windows. Systems with non-global zones, and particularly those with whole root zones, make this problem very much more severe.
A common reason given for this is that the tools update the contents file, {{{/var/sadm/install/contents}}}, in a naïve way: in particular that they rewrite the whole file for every change. I thought it would be interesting to check this, or to find out what the other reasons were for the catastrophic performance of these tools.
[[DTrace|http://www.sun.com/bigadmin/content/dtrace/]] is a very good tool for this sort of performance analysis. I was already almost sure, from having used tools like {{{iostat}}}, that the performance bottleneck was I/O, so I used a slightly modified version of {{{iofileb.d}}} from [[Brendan Gregg|http://www.brendangregg.com/]]'s excellent [[DTrace toolkit|http://www.brendangregg.com/dtrace.html#DTraceToolkit]]. The interesting part of this script is
{{{
io:::start
{
@files[pid, walltimestamp, execname, args[2]->fi_pathname] = sum(args[0]->b_bcount);
}
}}}
The differences to the original are that it records timestamp information, and it prints the results in a format more friendly to [[SQLite|http://www.sqlite.org/]], which I used to analyse the data. Once the data was imported into ~SQLite, I ran various queries on it to work out here the I/O was happening. Note that what is being measured is the //amount// of I/O: this is not an attempt to measure how well the I/O system is performing, which would be better understood by looking at I/O wait times and queue lengths, but would depend a lot on the hardware. Rather it's simply an attempt to understand where the patch tools do most of their I/O.
The results below were obtained on an x86 system running Solaris 10u4 with no non-global zones (unless otherwise specified). Other than the patches being added or removed, it was up to date with all the recommended and security patches at the time the tests were run. The machine was largely quiescent other than patch activity during the tests, but it was up multiuser. The results below are not in chronological order (I didn't want to start with an extreme example). All I/O figures are in kB, where 1kB is 1024B.
!125370-06 & 127112-05, 30 November 2007
These two patches were released shortly before this date, and were applied with PCA:
* 125370-06: "~SunOS 5.10_x86: Fault Manager patch". 10u4 shipped with 125370-05.
* 127112-05: "~SunOS 5.10_x86: kernel patch". Previous version installed was 127112-03.
Summary figures for I/O:
|noBorder topAlign|k
| Total I/0| 1,847,000| approximately 1.75GB|
| I/O to {{{/var/sadm/install/t.contents}}}| 1,429,892| approx 1.37GB|
| all other I/O| 417,108| approx 0.4GB|
From this we can see that about 77% of all I/O goes to a single file, which is the contents file (or a temporary copy of it). The mean I/O size was about 370k to this file, and about 49k to all other files. Only two programs read or wrote to this file:
|noBorder topAlign|k
| k| count|execname |
| 429,604| 4,183|{{{pkginstall}}} |
| 264| 8|{{{fsflush}}} | spurious, this is probably {{{pkginstall}}} |
| 24| 3|{{{removef}}} |
!125953-05 & 125953-06, 29 November 2007
These are two versions of the same fairly small patch: "Sun Java Web Console 3.0.4_x86: Support for Application Server 8.2 EE". 10u4 shipped with -04, and I wanted to see if there was a difference in performance dependent on whether there was a "real" previous patch. Summary I/O is as follows:
|noBorder topAlign|k
|>|patch 125953-05 |
| Total I/0| 207,154|
| I/O to {{{/var/sadm/install/t.contents}}}| 178,624|
| all other I/O| 28,530|
|>|patch 125953-06 |
| Total I/0| 264,483|
| I/O to {{{/var/sadm/install/t.contents}}}| 223,280|
| all other I/O| 41,203|
So, for 125953-05 about 86%, and for 125953-06 about 84% of all I/O goes to the contents file: there seems to be little difference apart from the later patch being rather larger. {{{pkginstall}}} generates all the I/O to the contents file.
!123591-07, 29 November 2007
This is "~SunOS 5.10_x86: ~PostgresSQL patch", and it's a monster. The system already had 123591-06 installed as a patch. Summary I/O for adding and removing thr patch is as follows:
|noBorder topAlign|k
|>|adding |
| Total I/0| 29,593,088|
| I/O to {{{/var/sadm/install/t.contents}}}| 29,242,232|
| all other I/O| 350,856|
|>|removing |
| Total I/0| 17,092,859|
| I/O to {{{/var/sadm/install/t.contents}}}| 16,823,728|
| all other I/O| 269,131|
For adding the patch about 99% of all I/O goes to the contents file. For removing it about 98% of all I/O is to the contents file. To put it another way: when adding the patch there is more than 80 //times// as much I/O to the contents file than all other I/O put together. And there is a spectacular amount of I/O: almost 30GB is read or written while adding this patch.
Again, only two programs did any I/O to the contents file while adding the patch:
|noBorder topAlign|k
| k| count|execname |
| 28,786,600| 1,286|{{{removef}}} |
| 446,616| 10|{{{pkginstall}}} |
| 8,856| 1|{{{fsflush}}} |spurious |
What can we say about this patch? The good news is that it's clearly unusual. The bad news is that this doesn't matter very much: a single patch like this in a bundle (and there have been two versions of this particular patch since 10u4 came out) will have a severe impact on patching time. This is the tactical nuclear weapon to the high explosive of the other patches.
!Conclusion
The picture drawn here is pretty grim: adding patches does between 4 and 90 times as much I/O to the contents file as it does to anything else. Removing patches is approximately the same story. Some patches in particular do a really spectacular amount of unneeded I/O.
The time to apply patches is problematic because for many organisations it is becoming hard to meet ~SLAs since patching takes so long.
This performance problem is already serious, and is likely to become extremely serious fairly soon. For systems with a significant number of whole-root zones, the performance problem is extremely serious already: applying 123591-07 to a system with 10 whole-root zones would result in approximately 1/3TB of I/O. A patch bundle might typically contain 100 patches of which (say) 2 were this bad: applying such a bundle might result in around 1TB of I/O for such a system. Such systems can not realistically be patched.
!What do do
There are two possible approaches.
# Wait for a not-yet-existing new patch system, to be released at some point with a new Solaris release. Discard all Solaris 10 and earlier systems at that point.
# Fix the problem with the current patch system, in the currently supported Solaris releases.
I hope it is clear that the first of these is not likely to have very good results for Sun.
Fortunately fixing the problem in the current patch system should not be very hard. Without inspecting the source it is hard to know the details of the problem, but the outline is clear: the contents file, which contains a line-based database of files installed on the system (a typical line is {{{/usr/bin/ls f none 0555 root bin 18720 35926 1168280662 SUNWcsu}}}, which says that {{{/usr/bin/ls}}} belongs to {{{SUNWcsu}}} and gives various information about the file) needs to be updated when files are changed. This updating is happening by rewriting the whole file each time. Unfortunately the contents file is large: on the system measured above, it contains 220,909 entries, and takes up 22MB. The result of this approach is that performance goes down approximately quadratically with the number of files on the system: an increase in the number of files will also increase the number of files patches change, resulting in more reads and writes of a larger file.
The contents file, at the moment, is essentially a database which is updated by reading and then writing the whole database for every change. An easy fix for the problem would be simply not to do this but adopt the following approach:
* while applying a patch anything that needs to update the file would not do so, but would instead record the update into an intent log;
* at the end of the patch, the intent log would be rolled into the contents file in one fell swoop, requiring one pass over the file.
There are two small complexities with this approach.
* Various different programs update the file and would need to know whether or not to write changes to it directly or to write to the log. This can easily be achieved, for instance, by having {{{patchadd}}} & {{{patchrm}}} set an environment variable which is listened to by these programs.
* If a patch application fails something needs to clean up the log: this probably simply means deleting it.
I'd suggest that this problem is addressed with some urgency: Sun's customers are already being severely impacted by it and it is only a matter of time before this problem becomes apparent to people who make purchase decisions.
/***
|''Name:''|LegacyStrikeThroughPlugin|
|''Description:''|Support for legacy (pre 2.1) strike through formatting|
|''Version:''|1.0.2|
|''Date:''|Jul 21, 2006|
|''Source:''|http://www.tiddlywiki.com/#LegacyStrikeThroughPlugin|
|''Author:''|MartinBudden (mjbudden (at) gmail (dot) com)|
|''License:''|[[BSD open source license]]|
|''CoreVersion:''|2.1.0|
***/
//{{{
// Ensure that the LegacyStrikeThrough Plugin is only installed once.
if(!version.extensions.LegacyStrikeThroughPlugin) {
version.extensions.LegacyStrikeThroughPlugin = {installed:true};
config.formatters.push(
{
name: "legacyStrikeByChar",
match: "==",
termRegExp: /(==)/mg,
element: "strike",
handler: config.formatterHelpers.createElementAndWikify
});
} //# end of "install only once"
//}}}
<<tagCloud excludeLists excludeSearch systemConfig meta>>
----
[[Notes]]
----
[[www.tfeb.org|http://www.tfeb.org/]]
[[www.cley.com|http://www.cley.com/]]
----
[[www.tiddlywiki.com|http://www.tiddlywiki.com/]]
/***
!Versions
!!Document
{{{$Id: //depot/www-tfeb-org/main/www-tfeb-org/html/notes/index.html#20 $}}}
!!~TiddlyWiki
|noBorder topAlign|k
|<<version>> |current |
|2.3.0 |20071213 |
|2.2.6 |20071111 |
|2.2.5 |20070912 |
|2.1.3 |20061126 |
|2.1.0 |20061003 |
|2.0.11 |20060616 |
|2.0.10 |20060423 |
|2.0.8 |20060413 |
|2.0.7 |20060403 |
!Shadows
<<list shadowed>>
!System configuration
***/
/*{{{*/
config.options.chkHttpReadOnly = true;
config.options.txtUserName = "Tim Bradshaw";
config.options.txtBackupFolder = "backup";
/*}}}*/
Sun have added a number of sanity checks to various dangerous disk-related commands in Solaris 10. This is a good thing, on the whole. Unfortunately they've failed to understand that, sometimes, you need to bypass the sanity checks. There are also some outright bugs in some of the sanity checks. my favourite being this (in 10u4):
{{{
$ swap -l
swapfile dev swaplo blocks free
/dev/dsk/c0d0s1 102,1 8 1220928 1220928
$ sudo swap -d /dev/dsk/c0d0s1
/dev/dsk/c0d0s1 was dump device --
invoking dumpadm(1M) -d swap to select new dump device
dumpadm: no swap devices are available
$ swap -l
No swap devices configured
$ grep c0d0s1 /etc/vfstab
/dev/dsk/c0d0s1 - - swap - no -
$ sudo swap -a /dev/dsk/c0d0s1
/dev/dsk/c0d0s1 is normally mounted on according to /etc/vfstab. Please remove this entry to use this device.
}}}
In other words: "you can't add this swap device, because it's specified as a swap device in {{{/etc/vfstab}}}". Instead you need to:
* edit {{{/etc/vfstab}}} to remove the reference to the device;
* swap on it;
* edit {{{/etc/vfstab}}} again to restore the reference.
Which is obviously not an improvement.
Well, this is just a bug, of course, and I suppose it will get fixed. But {{{format}}}, for instance, now prevents you labelling a disk if it thinks it might be in use. That's a good thing, except that //there is no way around this//. Sun don't seem to understand that there can be cases where the alternatives to doing something like this might be very expensive & disruptive indeed.
Except, of course, there is a way around this restriction. If you set the environment variable {{{NOINUSE_CHECK}}} to {{{1}}} (or probably any value), {{{format}}} & {{{swap}}} will not check.
{{{
$ sudo NOINUSE_CHECK=1 swap -a /dev/dsk/c0d0s1
operating system crash dump was previously disabled --
invoking dumpadm(1M) -d swap to select new dump device
$ sudo swap -l
swapfile dev swaplo blocks free
/dev/dsk/c0d0s1 102,1 8 1220928 1220928
}}}
It would, of course, be better if they checked, but allowed you to override the check, and better still if they documented this.
* [[NOINUSE_CHECK]]: a workaround for some recent stupidity in Solaris
* [[Solaris and OpenSolaris]]: it's just as confusing as it seems
* [[Allocated and reserved swap in Solaris]]: it's not as confusing as it seems
* [[Ethernet speed and duplex negotiation]]: how to avoid duplex mismatches
* [[Why separate /var is a good idea]]: it is not a good idea to configure important Solaris systems so that {{{/var}}} shares with {{{/}}}, despite recommendations to the contrary
* [[Initial performance measurement of the Solaris patch tools]]: some measurements with ~DTrace show how bad the situation is, and where the time is going
* [[Solaris 10 x86 boot archives and mirrored disks]]: some pointers, and a warning
* [[Security issues with HP network printers]]: what they are, and how to fix them
* [[Configuring a VNC server on Solaris]]
* [[VNC for system administrators]]
* [[Split keys with PGP]]: requiring a quorum of people to decrypt something
----
* [[A ZFS workaround for old IDE machines in Solaris 10]]: I believe this is no longer needed
HP make a lot of printers, and many of the larger ones are network connected. They contain firmware of course, and in the way of things that firmware is often buggy & needs to be upgraded. The way you upgrade the firmware is quite interesting: you download a new firmware image from HP, and send it to the printer with the normal print commands (on typical Unix systems, with lp or similar). The printer realises that what it's being asked to print is actually firmware, and instead of printing it installs the new image.
Well, of course, this is a disaster waiting to happen. First of all this means that anyone who can print to the printer (whether or not they have network access) can install new firmware. Secondly, unless the printer has some way of validating that what has been sent to it is actually something produced by HP (by verifying it was signed with an ~HP-owned private key, say), the firmware could be anything.
The first issue is a serious system administration problem - the last thing I want, as a sysadmin, is some user thinking that it would be smart to upgrade the printers I run.
The second issue is much more serious. Because, of course, these things are actually fully-fledged computers, attached to the network (the larger ones have disks). Anyone who can print to the device can, by default, install any program onto it. And, worst of all, a compromised printer doesn't even have to sniff the network for sensitive information: people will send it things for free. Lots of printers are used to print invoices, statements and other interesting things, and it's probably not very hard to peruse these for financial information on the way past.
HP are aware of this issue (although they describe the problem as being a possible denial-of-service attack: it's much, much worse than that), and it's possible to disable this upgrade facility using SNMP. But at the time of writing (January 2007) it is not disabled by default. This is unacceptable in my opinion.
This has been discussed on [[Bugtraq|http://www.securityfocus.com/bid/11297/]]. Here is a copy of HP's recommended fix.
<<<
{{{
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
HP SECURITY BULLETIN
HPSBPI01085 REVISION: 0
SSRT4840 rev. 0 HP LaserJet remote upgrade denial of service (DoS)
NOTICE:
There are no restrictions for distribution of this Bulletin
provided that it remains complete and intact.
The information in this Security bulletin should be acted upon
as soon as possible.
INITIAL RELEASE:
13 October 2004
POTENTIAL SECURITY IMPACT:
remote denial of service (DoS)
SOURCE:
HEWLETT-PACKARD COMPANY
HP Software Security Response Team
REFERENCES:
BID 11297
VULNERABILITY SUMMARY:
A potential security vulnerability has been identified with the
remote upgrade capability of Laserjet printers. This
vulnerability could be exploited remotely to create a denial of
service (DoS).
SUPPORTED SOFTWARE VERSIONS*: ONLY impacted versions are listed.
HP Color LaserJet 2500, Business InkJet 3000, LaserJet 3700,
LaserJet 4100mfp, LaserJet 4200, LaserJet 4300,
Color LaserJet 4600, Color LaserJet 4650, Color LaserJet 5500,
Color LaserJet 5550, LaserJet 9000, LaserJet 9000mfp,
LaserJet 9040mfp, LaserJet 9050, LaserJet 9050mfp, LaserJet 9055,
LaserJet 9065, Color LaserJet 9500, Color LaserJet 9500mfp.
BACKGROUND:
This issue has been reported here:
http://www.securityfocus.com/bid/11297
By default the remote upgrade capability of HP Laserjet printers
allows firmware upgrades without requiring a password. This could
be exploited to download malicious firmware which could result in
a denial of service.
RESOLUTION:
The ability to receive upgrade files for a printer can be
controlled through SNMP. Using the techniques described here,
securely configure the networked printer:
<http://h20000.www2.hp.com/bizsupport/TechSupport/
Document.jsp?objectID=bpj05999>
Then, disable the upgrade capability of the printer via SNMP,
setting the remote-upgrade-enable object to an integer value of 1:
1.3.6.1.4.1.11.2.3.9.4.2.1.1.20.8.0
remote-upgrade-enable OBJECT-TYPE
SYNTAX INTEGER {
eOff(1),
eOn(2)
}
ACCESS read-write
STATUS mandatory
DESCRIPTION
"Enables or disables the ability to upgrade the
firmware associated with any device that is
available to be upgraded. "
::= { upgradable-devices 8 }
Note: To perform an upgrade of the printer, this object must be
set to an integer value of 2. After the upgrade the object
should be set to an integer value of 1 to disable the
upgrade capability.
BULLETIN REVISION HISTORY:
Revision 0 - 13 October 2004
Initial release
* The software product category that this Security Bulletin
relates to is represented by the 5th and 6th characters of the
Bulletin number: GN=General, MA=Management Agents,
MI=Misc. 3rd party, MP=HP-MPE/iX, NS=HP NonStop Servers,
OV=HP OpenVMS, PI=HP Printing & Imaging, ST=HP Storage,
TU=HP Tru64 UNIX, TL=Trusted Linux, UX=HP-UX, VV=Virtual Vault
SUPPORT: For further information, contact HP Services support
channel.
SUBSCRIBE: To initiate a subscription to receive future HP
Security Bulletins via Email:
http://h30046.www3.hp.com/driverAlertProfile.php?regioncode
=NA&langcode=USENG&jumpid=in_SC-GEN__driverITRC&topiccode=ITRC
On the web page:
Driver and Support Alerts/Notifications Sign-up: Product Selection
Under Step1: your products
1. Select product category:
- a minimum of servers must be selected.
2. Select product family or search:
- a minimum of one product must be selected.
3. Add a product:
- a minimum of one product must be added.
In Step 2: your operating system(s)
- check ALL operating systems for which alerts are required.
Complete the form and Save.
To update an existing subscription:
http://h30046.www3.hp.com/subSignIn.php
Log in on the web page Subscriber's choice for Business: sign-in.
On the Web page: Subscriber's Choice: your profile summary
- - use Edit Profile to update appropriate sections.
Note: In addition to the individual alerts/notifications for the
selected operating systems/products, subscribers will
automatically receive one copy of alerts for non-operating system
categories (i.e., a subscriber who signs up for all six operating
system alerts will only receive one copy of all the non-operating
system alerts).
REPORT: To report a potential security vulnerability with any HP
supported product, send Email to: security-alert@hp.com. It is
strongly recommended that security related information being
communicated to HP be encrypted using PGP, especially exploit
information. To obtain the security-alert PGP key please send an
e-mail message to security-alert@hp.com with the Subject of
'get key' (no quotes).
System management and security procedures must be reviewed
frequently to maintain system integrity. HP is continually
reviewing and enhancing the security features of software products
to provide customers with current secure solutions.
"HP is broadly distributing this Security Bulletin in order to
bring to the attention of users of the affected HP products the
important security information contained in this Bulletin. HP
recommends that all users determine the applicability of this
information to their individual situations and take appropriate
action. HP does not warrant that this information is necessarily
accurate or complete for all user situations and, consequently,
HP will not be responsible for any damages resulting from user's
use or disregard of the information provided in this Bulletin.
To the extent permitted by law, HP disclaims all warranties,
either express or implied, including the warranties of
merchantability and fitness for a particular purpose, title
and non-infringement."
(c)Copyright 2004 Hewlett-Packard Development Company, L.P.
Hewlett-Packard Company shall not be liable for technical or
editorial errors or omissions contained herein. The information
provided is provided "as is" without warranty of any kind. To the
extent permitted by law, neither HP or its affiliates,
subcontractors or suppliers will be liable for incidental, special
or consequential damages including downtime cost; lost profits;
damages relating to the procurement of substitute products or
services; or damages for loss of data, or software restoration.
The information in this document is subject to change without
notice. Hewlett-Packard Company and the names of Hewlett-Packard
products referenced herein are trademarks of Hewlett-Packard
Company in the United States and other countries. Other product
and company names mentioned herein may be trademarks of their
respective owners.
-----BEGIN PGP SIGNATURE-----
Version: PGP 8.0.2
iQA/AwUBQW5xOuAfOvwtKn1ZEQJyjgCffBL5BAOS10FZxe2PmAptf0eK9cQAnib4
J9nl1HVdO9ifHQxPpVGWpTEW
=FY6J
-----END PGP SIGNATURE-----
}}}
<<<
half a page of scribbled lines
http://www.tfeb.org/notes/
Solaris 10 u1, alternatively known as Solaris 10 1/06 & later releases use [[GRUB|http://www.gnu.org/software/grub/]] to boot on x86 systems, replacing the antiquated Solaris x86 boot system which was still used in the original 3/05 release. As of this writing, the manuals at [[docs.sun.com|http://docs.sun.com/]] are somewhat confused and unless you look quite hard you still get a description of the previous boot system. There is some good information at [[BigAdmin|http://www.sun.com/bigadmin/home/index.html]]:
* [[GRUB and the Solaris 10 1/06 OS: The New Bootloader for x86 Platforms|http://www.sun.com/bigadmin/features/articles/grub_boot_solaris.html]];
* [[FAQ for GRUB and the Solaris 10 1/06 OS: The New Bootloader for x86 Platforms|http://www.sun.com/bigadmin/features/articles/grub_boot_faq.html]].
To anyone used to SPARC systems, it's clear that booting x86 machines is inherently a grotesquely unpleasant process, made harder by the common desire to be able to boot several different and mutually hostile ~OSs. Crucial to booting Solaris with GRUB is the boot archive, which normally lives in {{{/platform/i86pc/boot_archive}}}. This is typically (always?) a {{{gzip}}}ped UFS filesystem image, and is loaded into memory by {{{multiboot}}}. One the kernel starts, it uses this image to load modules and so on without needing to do I/O. Eventually it will gain the ability to do I/O to the root disk, and will mount the real root and continue from there.
The equivalent process on SPARC systems involves {{{ufsboot}}} loading the kernel and handing over control to it. The kernel then initially calls back into {{{ufsboot}}} to load modules from the disk until it reaches the point where it can do I/O itself. This is a much more robust process because there is no danger that the contents of the disk might be out of sync with the contents of the boot archive. Presumably it's not possible on x86 systems due to some architectural shortcoming.
On x86 systems it's important that the boot archive be kept up to date if things get updated – for instance if new versions of kernel modules appear, they need to be consistent between the boot archive and the disk. The {{{bootadm}}} program does this, and ought to be called when patches are installed. If something happens which leaves the boot archive out of date – I think this is generally the result of patches failing – then, on boot, the machine will tell you to choose GRUB's "Solaris failsafe" option and follow the instructions to update the boot archive.
If you have a root filesystem mirrored using SVM, then doing this will almost certainly cause significant corruption to the root filesystem on the machine, and will probably need to do a ground-up reinstall / recovery. What happens is as follows.
* When you choose the failsafe option, it uses a special boot archive, which is {{{/boot/x86.miniroot-safe}}}, which is essentially a traditional miniroot designed for recovery or installation.
* Some script from this miniroot ({{{/sbin/install-recovery}}} in fact) is run by SMF which attempts to work out if the boot archive is out of date by grovelling over the filesystems it finds. It attempts to work out if the disks are mirrored by looking at vfstab. If they are it doesn't try and fix the archives but instead says: <html>
<blockquote><pre>
To manually recover the boot archive on a root mirror,
mount the first side (the one that the system boots from)
and run:
bootadm update-archive -R <mount_point>
</pre></blockquote></html> That is all it says.
What happens if you do this?
* You mount half the mirror and run {{{bootadm}}} on it.
* This updates the archive, and makes significant modifications to the UFS structure on that half of the mirror as a result, which changes are not reflected on the other half of the mirror.
* You then reboot the machine: GRUB reads the boot archive, and everything looks good. The kernel starts, loads the SVM modules and //mounts the mirror as {{{/}}}//.
* The world is now nearly over: in the normal way, reads come from either half of the mirror, and writes go to both. But the two halves of the mirror have different contents since you modified one, following instructions above. So random incoherent crap is read from the filesystem, and (worse) written back to it.
* In fairly short order neither half of the mirror has a consistent UFS on it. The inevitable bad things follow.
If you catch this in time you can recover by:
* booting single user;
* detaching one half of the mirror (forcibly if need be);
* booting from the failsafe GRUB option;
* running {{{fsck}}} on the device corresponding to the remaining mirror half.
If you're lucky you'll be able to reconstruct the filesystem. Finally reboot, and reattach the detached submirror.
Of course, the problem is that {{{/sbin/install-recovery}}} in the miniroot is telling you something entirely wrong. What you should do is:
* when the machine initially boots and complains about the out of date boot archive, break the mirror //before// trying to update the boot archive by doing the failsafe boot – you may need to use {{{metasync}}} to synchronise the mirror first;
* if the boot archive is so badly mangled that the machine won't boot, then after doing a failsafe boot you need to do significantly more work to get the machine up off one half of the mirror – edit {{{/etc/system}}} and {{{/etc/vfstab}}} in the good half mirror at least).
----
This information is correct as of Solaris 10 u5 – with luck the instructions & process will be improved soon.
Even by Sun's standards, the confusion around ~OpenSolaris has been impressive, with an ever-changing list of project names and releases.
Solaris-related notes here generally refer to the version of "commercial Solaris" that was current when they were written. I've tried to indicate when that is not true, and when notes have become obsolete.
!Zones and containers
Sun are somewhat confused about what they call things, and in particular the terms "zone" and "container" have not always been used consistently. I suspect there is confusion even within the non-marketing parts of Sun about the proper terminology (the marketing terminology can be more-or-less dismissed as things get renamed every few weeks to cause maximum fear uncertainty and doubt among customers).
The 10u4 reference manual describes containers as follows.
<<<
A Solaris Container is a complete runtime environment for applications. Solaris 10 Resource manager and Solaris Zones software partitioning technology are both parts of the container. These components address different qualities the container can deliver and work together to create a complete container. The zones portion of the container provides a virtual mapping from the application to the platform resources. Zones allow application components to be isolated from one another even though the zones share a single instance of the Solaris Operating System. Resource management features permit you to allocate the quantity of resources that a workload receives.
<<<
A reasonable set of definitions might be:
* a ''container'' provides a runtime environment for applications and contains one or both of ''resource management'' and a ''zone'';
* ''resource management'' provides constraint on various resources – such as memory, processor time, number of processors – that an application may use;
* a ''zone'' provides isolation for applications, and specifically provides various isolated namespaces, for instance the spaces of file names and process identifiers.
Sun would probably insist that a container must include a zone, but that seems unnecessarily lopsided to me.
Solaris has an increasingly sophisticated resource management framework dating back well before Solaris 10, but, at the time of writing (late 2007), it is not particularly well integrated and not entirely complete (there is no control over I/O; often a critical resource for larger systems). The zones system is far more integrated and coherent.
!What zones are
Zones are essentially extremely souped-up {{{chroot}}} environments: they provide a little world which looks, superficially like a stand-alone Solaris system, but in fact is merely, well, a zone, sitting on top of a host OS.
* Zones have their own root directories and their own device trees. Depending on configuration, a zone may share much of the standard Solaris directory structure ({{{/usr}}} &c) with the "global zone" – the parent machine – via loopback mounts. A zone which does this is called a "sparse" zone, and uses up rather little storage: how much depends on various factors, including obscure things like how many patches have been applied to the parent, since patch backouts are copied (this is probably a bug), but it is ~100MB. All the filespace which is shared with the global zone is obviously readonly in the sparse zone. At the other end of the scale are "whole root" zones, which have copies of {{{/usr}}} &c, and use a lot more filespace. The advantage of whole root zones is that the contents of {{{/usr}}} &c can be changed within the zone; the disadvantage is much more filespace usage, and more memory usage (executable pages can not be shared since they are backed by different files).
* Zones have their own copy of {{{init}}} and the entire tree of processes descending from it.
/***
|''Name:''|SparklinePlugin|
|''Description:''|Sparklines macro|
***/
//{{{
if(!version.extensions.SparklinePlugin) {
version.extensions.SparklinePlugin = {installed:true};
//--
//-- Sparklines
//--
config.macros.sparkline = {};
config.macros.sparkline.handler = function(place,macroName,params)
{
var data = [];
var min = 0;
var max = 0;
var v;
for(var t=0; t<params.length; t++) {
v = parseInt(params[t]);
if(v < min)
min = v;
if(v > max)
max = v;
data.push(v);
}
if(data.length < 1)
return;
var box = createTiddlyElement(place,"span",null,"sparkline",String.fromCharCode(160));
box.title = data.join(",");
var w = box.offsetWidth;
var h = box.offsetHeight;
box.style.paddingRight = (data.length * 2 - w) + "px";
box.style.position = "relative";
for(var d=0; d<data.length; d++) {
var tick = document.createElement("img");
tick.border = 0;
tick.className = "sparktick";
tick.style.position = "absolute";
tick.src = "data:image/gif,GIF89a%01%00%01%00%91%FF%00%FF%FF%FF%00%00%00%C0%C0%C0%00%00%00!%F9%04%01%00%00%02%00%2C%00%00%00%00%01%00%01%00%40%02%02T%01%00%3B";
tick.style.left = d*2 + "px";
tick.style.width = "2px";
v = Math.floor(((data[d] - min)/(max-min)) * h);
tick.style.top = (h-v) + "px";
tick.style.height = v + "px";
box.appendChild(tick);
}
};
}
//}}}
Something that you might want to do is to arrange life so that you need a //quorum// of people to perform some operation. The classic example is launching ~ICBMs, where life is (or once was) arranged so that two people would need to turn two keys simultaneously, with enough separation between the locks that one person could not do it alone. A similar case nowadays might be access to a critical cryptographic key - perhaps a key which controls access to sensitive financial information such as credit card numbers.
For a cryptographic system the requirement is something like the following.
<<<
From a population of //n// people, a subset of at least //m// should be required to access some bit of information. Any subset of at least //m// people from //n// should be able to do this.
<<<
Here's a way of achieving this with PGP (in fact, I use [[GPG|http://www.gnupg.org/]]). There is nothing clever about this, and I am not at all sure it is even a very good way of doing it: but it is //a// way of doing it which I think works reasonably well. I haven't read the literature on this though, so I may be missing something trivial: I'd be interested in knowing if I have. The final advantage of this technique is that it very simple to implement.
GPG won't do what we need directly, so what we need to do is use what it //can// do. One thing it can do is to encrypt something so that any one of a number keys will decrypt it - this is important for mail to multiple recipients for instance, where you want to encrypt the mail with the public keys of all the recipients, so that any one of them can decrypt it. This itself is one limiting case of the //m// of //n// problem, although there is a simpler answer if you don't mind everyone sharing a key pair, which we wouldn't.
So the trick is to give each of the combinations of people required for a quorum a key, and then to split that key between the members of the combination. So for instance if we had 3 people, Alice, Bob, and Charles, and we required two of them to be present to decrypt something, we'd generate keys as follows:
* alice-bob;
* alice-charles;
* bob-charles.
alice would then have half of the alice-bob and alice-charles keys and so on.
There are some issues with this:
* how many keys might there be?
* how badly are the keys weakened - for instance if part of one key leaks out how much does that weaken the rest of it?
The first answer is simple: in general there might be rather a lot of keys, but in the normal case the number will not be too bad. The number of combinations of //r// from //n// is often written as //n//C//r// (`//n// choose //r//'), and:
<<<
//n//C//r// = //n//! / //r//!(//n// - //r//)!
<<<
So, if you had 50 people and you required a quorum of 30 to decrypt something, you're in a fairly bad way: you need 50!/(30!*20!) keys, which is about 5e13. But a much more likely case would be 50 people and a quorum of 2 or 3. The general answer for a quorum of 2 from //n// is:
<<<
//n//C2 = //n//! / 2(//n//-2)! = //n//(//n// - 1) / 2
<<<
So for 50 this is 1225 keys, which is manageable. Each person is in 49 combinations (them and one other person), so there are 49 partial keys per person, which is fine.
For a quorum of 3 the number of keys is:
<<<
//n//C3 = //n//! / 6(//n// - 3)! = //n//(//n// - 1)(//n// - 2) / 6
<<<
So for 50 people this is 19600 keys which is OK. Each person has quite a lot of partial keys though - 49C2 or 1176. This is still easily within the realm of a USB key though.
How much are keys weakened by this? The worst case, I think, is the 2-person quorum. In this case everyone has half of all the keys in which they partake. If someone's partial keys leak, then an attacker has half the bits of those keys, and they also have //n// - 1 keys to attack. That's probably not catastrophic, so long as the keys were long enough to start with, but it would obviously be sensible to regenerate the keys in this case. If //two// people's keys leak then an attacker has all of one key of course, and this is bad.
There are various things that can be done on top of a scheme like this - for instance requiring a quorum of 2 //and// one of a smaller number of people's keys (perhaps any 2 people and one person who is a director). This can be done by simply composing schemes like the above - the information is encrypted first so any director can decrypt it, and then so any one of the combinations can decrypt it.
In practice it's probably better to split keys by picking every //n//^^th^^ byte from them as this avoids any issue with people getting just header or checksum bits from keys.
/***
!Printing
Suppress hover menu and header in printed output
***/
/*{{{*/
@media print {
#hoverMenu, div.header {display: none ! important;}
}
/*}}}*/
/***
!Mucking around with tables
***/
/*{{{*/
/* Table with no overall border */
table.noBorder {
border-style: hidden !important;
}
/* Table with no borders at all */
table.noBorders *, table.noborders {
border-style: hidden !important;
}
/* Table with rows top aligned */
table.topAlign tr {
vertical-align: top ! important;
}
/*}}}*/
/***
''Plugin:'' Tag Cloud Macro
''Author:'' Clint Checketts
''Source URL: http://checkettsweb.com/styles/themes.htm''
!Usage
<<tagCloud>>
!Code
***/
//{{{
version.extensions.tagCloud = {major: 1, minor: 0 , revision: 0, date: new Date(2006,2,04)};
//Created by Clint Checketts, contributions by Jonny Leroy and Eric Shulman
config.macros.tagCloud = {
noTags: "No tag cloud created because there are no tags.",
tooltip: "%1 tiddlers tagged with '%0'"
};
config.macros.tagCloud.handler = function(place,macroName,params) {
var tagCloudWrapper = createTiddlyElement(place,"div",null,"tagCloud",null);
var tags = store.getTags();
for (var t=0; t<tags.length; t++) {
for (var p=0;p<params.length; p++) if (tags[t][0] == params[p]) tags[t][0] = "";
}
if(tags.length == 0)
createTiddlyElement(tagCloudWrapper,"span",null,null,this.noTags);
//Findout the maximum number of tags
var mostTags = 0;
for (var t=0; t<tags.length; t++) if (tags[t][0].length > 0){
if (tags[t][1] > mostTags) mostTags = tags[t][1];
}
//divide the mostTags into 4 segments for the 4 different tagCloud sizes
var tagSegment = mostTags / 4;
for (var t=0; t<tags.length; t++) if (tags[t][0].length > 0){
var tagCloudElement = createTiddlyElement(tagCloudWrapper,"span",null,null,null);
tagCloudWrapper.appendChild(document.createTextNode(" "));
var theTag = createTiddlyButton(tagCloudElement,tags[t][0],this.tooltip.format(tags[t]),onClickTag,"tagCloudtag tagCloud" + (Math.round(tags[t][1]/tagSegment)+1));
theTag.setAttribute("tag",tags[t][0]);
}
};
setStylesheet(".tagCloud span{height: 1.8em;margin: 3px;}.tagCloud1{font-size: 1.2em;}.tagCloud2{font-size: 1.4em;}.tagCloud3{font-size: 1.6em;}.tagCloud4{font-size: 1.8em;}.tagCloud5{font-size: 1.8em;font-weight: bold;}","tagCloudsStyles");
//}}}
|noBorder topAlign|k
| e|//tfb at tfeb.org// |
| w|[[www.tfeb.org|http://www.tfeb.org/]] |
| w|[[www.cley.com|http://www.cley.com/]] |
VNC is a wonderful tool for system administrators, which doesn't see enough use. People who look after machines often need to do it from many locations, including home, office, datacentre and so on. And they run long-running interactive applications which have significant state, which don't run on their desktop or laptop. Some of this can be dealt with by things like the `screen' program, which allows you to detach and reattach to tty sessions. But this doesn't really cut it for graphical applications, and plenty of applications systems people run nowadays are such. VNC is a good answer to that problem.
For Unix admins, a typical graphical desktop session uses some flavour of X server, and the connections look roughly like the first picture.
[>img[X11 connections|figures/X11.png]]
Here the clients are the programs you actually run, and they have connections to an X server which runs your screen, keyboard and mouse, and which, critically, runs on the machine you actually type at. These connections have state – in particular the X server holds state for the clients. Very few, if any, clients can reestablish connections after they've been lost. Finally, they're connections to a //particular// X server. All this means that all these clients will not survive when you, say, take your laptop and its X server home. Most people don't worry about this: they just run all their applications locally, and this is fine. But if the application you are running is on a ~SF15K you can't really take it home.
VNC essentially replaces the first picture with the second one.
[>img[Xvnc connections|figures/Xvnc.png]]
The various clients still talk to an X server. But the X server has been cut in half: instead of looking after a screen directly, it now talks the VNC protocol, which is a protocol for shipping low-level images of a screen around in a reasonably efficient way, and handling mouse and keyboard input coming the other way. The good thing about VNC is that it is essentially //stateless//: the VNC server doesn't mind if the far end of the connection goes away, it will just reestablish a connection (shipping over a new picture of the screen) in due course. And the clients never know anything about this.
So what you can do is to run the VNC server (Xvnc in one of its various forms) on a machine which is always visible from the machines you need to manage. You can then run the VNC viewer anywhere you like, or you can run different VNC viewers on machines at work, at home and so on. There's even a Java applet VNC viewer so you can run the whole thing in a web browser. All you need to be able to do is get a TCP stream to the VNC server, and you can do the usual tricks of tunnelling this through ssh (and ssh over a VPN) top make sure it's secure. And your desktop session can now follow you wherever you are.
Of course this looks like it should be terribly slow. But from the perspective of a systems person it's not too bad, as the VNC protocol does a number of things to minimise the data that gets shipped around (a single-colour background helps here), and broadband connections are now acceptably fast. You probably wouldn't want to run video games over it, or run the whole thing over a GPRS connection, but it's really fine. Latency can be an issue if you're too many network hops away, causing unpleasant `dragginess', but even that is liveable-with I find.
There is a strand of thought which says that, as filesystems, {{{/var}}} should be in {{{/}}}: this is Sun's current recommendation. It is also often said that people who disagree with this are doing so basically for reasons of tradition, with the implication that these people are old-fashioned and inflexible in their thinking. This is not the case: the truth is that putting {{{/var}}} in {{{/}}} is naïve, and the people who are not doing this are those who have actually thought about the problem.
{{{/}}}, (specifically {{{/etc}}}), contains critical quasi-static data: files which, if they are damaged, will cause the system to fail to boot correctly, but which are updated occasionally. Examples include the password & shadow files, {{{/etc/vfstab}}} and so on.
{{{/var}}} contains highly dynamic data, which data can be created by any user or application on the system – both by creating files in {{{/var/tmp}}}, but also by logging via syslog &c.
So any user or application on the system can fill the filesystem where {{{/var}}} lives. If this is {{{/}}} then any user or application can fill {{{/}}}.
OK, so consider the files in {{{/etc}}}. In order to avoid a disaster we need to be able to guarantee that any update to these files will either succeed, or will make no modification to the files. Under no circumstances must the files be left in a part-updated state (for instance a truncated {{{/etc/passwd}}} or {{{/etc/shadow}}} is very bad news indeed). This must hold even in the case where the file system fills half-way through the update.
It is possible that Sun can guarantee this for all Sun-provided applications. My guess is that this is the basis on which the EIS standard has been created.
However that is not nearly sufficient, because it is not only Sun's applications which update these files. Any customer or ISV can write an application which, for instance, updates the password & shadow files. It is extremely unlikely that most of these applications are robust in the event of the filesystem filling as they run. Sun can never warrant these applications.
A hard-line answer would be that it is up to the customer/ISV to warrant their own applications. I probably don't have to say why this is a bad idea. A better answer (and the only acceptable one) is to make sure that unbounded space usage in {{{/var}}} can not fill {{{/}}}, and the way to do this is by having them in distinct filesystems. Obviously it is still possible for {{{/}}} to fill, but this will reduce the likelyhood of this happening, and thus reduce the risk.
In fact, Sun can not almost certainly not warrant even their own applications: in at least some cases the documented way to change files in {{{/etc}}} is with a text editor. An example of this is {{{/etc/vfstab}}}. To warrant this, Sun would need to either specify a known-safe text editor (vi is not safe in this sense), or warrant all the text editors that exist or could ever exist. Neither of these is a good option.
!Notes
* Obviously this does not apply to small, simple, or very non-critical systems. Configuring such systems with more than the absolute minimum number of filesystems is cearly a waste of time and effort: I currently build systems like that with a single UFS {{{/}}} and a ZFS pool for user filesystems. When ZFS root filesystems are available, I will probably use a single pool for such machines and not bother with distinct system filesystems within the pool.
* For other systems, these considerations will still apply when ZFS root filesystems become the norm. {{{/var}}} and {{{/}}} will need to be separate filesystems, with either a quota for {{{/var}}} or (better) a reservation for {{{/}}}. Of course, these filesystems can coexist within the same pool.
* Even in the case where critical data is not put at risk, having {{{/var}}} in {{{/}}} allows what is essentially a denial-of-service attack: if {{{/}}} is full then passwords can not be changed, users can not be added &c &c, and any user or application can arrange for this to happen. That is very poor design.
* A general design rule would be to always keep dynamic and quasi-static data in disjoint domains, especially where there is no control over write access to the dynamic data.
/***
|''Name:''|YourSearchPlugin|
|''Version:''|2.1.3 (2008-04-16)|
|''Source:''|http://tiddlywiki.abego-software.de/#YourSearchPlugin|
|''Author:''|UdoBorkowski (ub [at] abego-software [dot] de)|
|''Licence:''|[[BSD open source license (abego Software)|http://www.abego-software.de/legal/apl-v10.html]]|
|''Copyright:''|© 2005-2008 [[abego Software|http://www.abego-software.de]]|
|''~CoreVersion:''|2.1.0|
|''Community:''|[[del.icio.us|http://del.icio.us/post?url=http://tiddlywiki.abego-software.de/index.html%23YourSearchPlugin]]|
|''Browser:''|Firefox 1.0.4+; Firefox 1.5; ~InternetExplorer 6.0|
!About YourSearch
YourSearch gives you a bunch of new features to simplify and speed up your daily searches in TiddlyWiki. It seamlessly integrates into the standard TiddlyWiki search: just start typing into the 'search' field and explore!
For more information see [[Help|YourSearch Help]].
!Compatibility
This plugin requires TiddlyWiki 2.1.
Check the [[archive|http://tiddlywiki.abego-software.de/archive]] for ~YourSearchPlugins supporting older versions of TiddlyWiki.
!Source Code
***/
/***
This plugin's source code is compressed (and hidden). Use this [[link|http://tiddlywiki.abego-software.de/archive/YourSearchPlugin/Plugin-YourSearch-src.2.1.3.js]] to get the readable source code.
***/
///%
if(!version.extensions.YourSearchPlugin){version.extensions.YourSearchPlugin={major:2,minor:1,revision:3,source:"http://tiddlywiki.abego-software.de/#YourSearchPlugin",licence:"[[BSD open source license (abego Software)|http://www.abego-software.de/legal/apl-v10.html]]",copyright:"Copyright (c) abego Software GmbH, 2005-2008 (www.abego-software.de)"};if(!window.abego){window.abego={};}
if(!Array.forEach){Array.forEach=function(_1,_2,_3){for(var i=0,len=_1.length;i<len;i++){_2.call(_3,_1[i],i,_1);}};Array.prototype.forEach=function(_5,_6){for(var i=0,len=this.length;i<len;i++){_5.call(_6,this[i],i,this);}};}
abego.toInt=function(s,_9){if(!s){return _9;}
var n=parseInt(s);return(n==NaN)?_9:n;};abego.createEllipsis=function(_b){var e=createTiddlyElement(_b,"span");e.innerHTML="…";};abego.shallowCopy=function(_d){if(!_d){return _d;}
var _e={};for(var n in _d){_e[n]=_d[n];}
return _e;};abego.copyOptions=function(_10){return!_10?{}:abego.shallowCopy(_10);};abego.countStrings=function(_11,s){if(!s){return 0;}
var len=s.length;var n=0;var _15=0;while(1){var i=_11.indexOf(s,_15);if(i<0){return n;}
n++;_15=i+len;}
return n;};abego.getBracedText=function(_17,_18,_19){if(!_18){_18=0;}
var re=/\{([^\}]*)\}/gm;re.lastIndex=_18;var m=re.exec(_17);if(m){var s=m[1];var _1d=abego.countStrings(s,"{");if(!_1d){if(_19){_19.lastIndex=re.lastIndex;}
return s;}
var len=_17.length;for(var i=re.lastIndex;i<len&&_1d;i++){var c=_17.charAt(i);if(c=="{"){_1d++;}else{if(c=="}"){_1d--;}}}
if(!_1d){if(_19){_19.lastIndex=i-1;}
return _17.substring(m.index+1,i-1);}}};abego.select=function(_21,_22,_23,_24){if(!_24){_24=[];}
_21.forEach(function(t){if(_22.call(_23,t)){_24.push(t);}});return _24;};abego.consumeEvent=function(e){if(e.stopPropagation){e.stopPropagation();}
if(e.preventDefault){e.preventDefault();}
e.cancelBubble=true;e.returnValue=true;};abego.TiddlerFilterTerm=function(_27,_28){if(!_28){_28={};}
var _29=_27;if(!_28.textIsRegExp){_29=_27.escapeRegExp();if(_28.fullWordMatch){_29="\\b"+_29+"\\b";}}
var _2a=new RegExp(_29,"m"+(_28.caseSensitive?"":"i"));this.tester=new abego.MultiFieldRegExpTester(_2a,_28.fields,_28.withExtendedFields);};abego.TiddlerFilterTerm.prototype.test=function(_2b){return this.tester.test(_2b);};abego.parseNewTiddlerCommandLine=function(s){var m=/(.*?)\.(?:\s+|$)([^#]*)(#.*)?/.exec(s);if(!m){m=/([^#]*)()(#.*)?/.exec(s);}
if(m){var r;if(m[3]){var s2=m[3].replace(/#/g,"");r=s2.parseParams("tag");}else{r=[[]];}
var _30=m[2]?m[2].trim():"";r.push({name:"text",value:_30});r[0].text=[_30];return{title:m[1].trim(),params:r};}else{return{title:s.trim(),params:[[]]};}};abego.parseTiddlerFilterTerm=function(_31,_32,_33){var re=/\s*(?:(?:\{([^\}]*)\})|(?:(=)|([#%!])|(?:(\w+)\s*\:(?!\/\/))|(?:(?:("(?:(?:\\")|[^"])+")|(?:\/((?:(?:\\\/)|[^\/])+)\/)|(\w+\:\/\/[^\s]+)|([^\s\)\-\"]+)))))/mg;var _35={"!":"title","%":"text","#":"tags"};var _36={};var _37;re.lastIndex=_32;while(1){var i=re.lastIndex;var m=re.exec(_31);if(!m||m.index!=i){throw"Word or String literal expected";}
if(m[1]){var _3a={};var _3b=abego.getBracedText(_31,0,_3a);if(!_3b){throw"Invalid {...} syntax";}
var f=Function("tiddler","return ("+_3b+");");return{func:f,lastIndex:_3a.lastIndex,markRE:null};}
if(m[2]){_37=true;}else{if(m[3]){_36[_35[m[3]]]=1;}else{if(m[4]){_36[m[4]]=1;}else{var _3d=m[6];var _3e=m[5]?window.eval(m[5]):m[6]?m[6]:m[7]?m[7]:m[8];var _33=abego.copyOptions(_33);_33.fullWordMatch=_37;_33.textIsRegExp=_3d;var _3f=[];for(var n in _36){_3f.push(n);}
if(_3f.length==0){_33.fields=_33.defaultFields;}else{_33.fields=_3f;_33.withExtendedFields=false;}
var _41=new abego.TiddlerFilterTerm(_3e,_33);var _42=_3d?_3e:_3e.escapeRegExp();if(_42&&_37){_42="\\b"+_42+"\\b";}
return{func:function(_43){return _41.test(_43);},lastIndex:re.lastIndex,markRE:_42?"(?:"+_42+")":null};}}}}};abego.BoolExp=function(s,_45,_46){this.s=s;var _47=_46&&_46.defaultOperationIs_OR;var _48=/\s*(?:(\-|not)|(\())/gi;var _49=/\s*\)/g;var _4a=/\s*(?:(and|\&\&)|(or|\|\|))/gi;var _4b=/\s*[^\)\s]/g;var _4c=/\s*(\-|not)?(\s*\()?/gi;var _4d;var _4e=function(_4f){_4c.lastIndex=_4f;var m=_4c.exec(s);var _51;var _52;if(m&&m.index==_4f){_4f+=m[0].length;_51=m[1];if(m[2]){var e=_4d(_4f);_49.lastIndex=e.lastIndex;if(!_49.exec(s)){throw"Missing ')'";}
_52={func:e.func,lastIndex:_49.lastIndex,markRE:e.markRE};}}
if(!_52){_52=_45(s,_4f,_46);}
if(_51){_52.func=(function(f){return function(_55){return!f(_55);};})(_52.func);_52.markRE=null;}
return _52;};_4d=function(_56){var _57=_4e(_56);while(1){var l=_57.lastIndex;_4a.lastIndex=l;var m=_4a.exec(s);var _5a;var _5b;if(m&&m.index==l){_5a=!m[1];_5b=_4e(_4a.lastIndex);}else{try{_5b=_4e(l);}
catch(e){return _57;}
_5a=_47;}
_57.func=(function(_5c,_5d,_5e){return _5e?function(_5f){return _5c(_5f)||_5d(_5f);}:function(_60){return _5c(_60)&&_5d(_60);};})(_57.func,_5b.func,_5a);_57.lastIndex=_5b.lastIndex;if(!_57.markRE){_57.markRE=_5b.markRE;}else{if(_5b.markRE){_57.markRE=_57.markRE+"|"+_5b.markRE;}}}};var _61=_4d(0);this.evalFunc=_61.func;if(_61.markRE){this.markRegExp=new RegExp(_61.markRE,_46.caseSensitive?"mg":"img");}};abego.BoolExp.prototype.exec=function(){return this.evalFunc.apply(this,arguments);};abego.BoolExp.prototype.getMarkRegExp=function(){return this.markRegExp;};abego.BoolExp.prototype.toString=function(){return this.s;};abego.MultiFieldRegExpTester=function(re,_63,_64){this.re=re;this.fields=_63?_63:["title","text","tags"];this.withExtendedFields=_64;};abego.MultiFieldRegExpTester.prototype.test=function(_65){var re=this.re;for(var i=0;i<this.fields.length;i++){var s=store.getValue(_65,this.fields[i]);if(typeof s=="string"&&re.test(s)){return this.fields[i];}}
if(this.withExtendedFields){return store.forEachField(_65,function(_69,_6a,_6b){return typeof _6b=="string"&&re.test(_6b)?_6a:null;},true);}
return null;};abego.TiddlerQuery=function(_6c,_6d,_6e,_6f,_70){if(_6e){this.regExp=new RegExp(_6c,_6d?"mg":"img");this.tester=new abego.MultiFieldRegExpTester(this.regExp,_6f,_70);}else{this.expr=new abego.BoolExp(_6c,abego.parseTiddlerFilterTerm,{defaultFields:_6f,caseSensitive:_6d,withExtendedFields:_70});}
this.getQueryText=function(){return _6c;};this.getUseRegExp=function(){return _6e;};this.getCaseSensitive=function(){return _6d;};this.getDefaultFields=function(){return _6f;};this.getWithExtendedFields=function(){return _70;};};abego.TiddlerQuery.prototype.test=function(_71){if(!_71){return false;}
if(this.regExp){return this.tester.test(_71);}
return this.expr.exec(_71);};abego.TiddlerQuery.prototype.filter=function(_72){return abego.select(_72,this.test,this);};abego.TiddlerQuery.prototype.getMarkRegExp=function(){if(this.regExp){return"".search(this.regExp)>=0?null:this.regExp;}
return this.expr.getMarkRegExp();};abego.TiddlerQuery.prototype.toString=function(){return(this.regExp?this.regExp:this.expr).toString();};abego.PageWiseRenderer=function(){this.firstIndexOnPage=0;};merge(abego.PageWiseRenderer.prototype,{setItems:function(_73){this.items=_73;this.setFirstIndexOnPage(0);},getMaxPagesInNavigation:function(){return 10;},getItemsCount:function(_74){return this.items?this.items.length:0;},getCurrentPageIndex:function(){return Math.floor(this.firstIndexOnPage/this.getItemsPerPage());},getLastPageIndex:function(){return Math.floor((this.getItemsCount()-1)/this.getItemsPerPage());},setFirstIndexOnPage:function(_75){this.firstIndexOnPage=Math.min(Math.max(0,_75),this.getItemsCount()-1);},getFirstIndexOnPage:function(){this.firstIndexOnPage=Math.floor(this.firstIndexOnPage/this.getItemsPerPage())*this.getItemsPerPage();return this.firstIndexOnPage;},getLastIndexOnPage:function(){return Math.min(this.getFirstIndexOnPage()+this.getItemsPerPage()-1,this.getItemsCount()-1);},onPageChanged:function(_76,_77){},renderPage:function(_78){if(_78.beginRendering){_78.beginRendering(this);}
try{if(this.getItemsCount()){var _79=this.getLastIndexOnPage();var _7a=-1;for(var i=this.getFirstIndexOnPage();i<=_79;i++){_7a++;_78.render(this,this.items[i],i,_7a);}}}
finally{if(_78.endRendering){_78.endRendering(this);}}},addPageNavigation:function(_7c){if(!this.getItemsCount()){return;}
var _7d=this;var _7e=function(e){if(!e){var e=window.event;}
abego.consumeEvent(e);var _80=abego.toInt(this.getAttribute("page"),0);var _81=_7d.getCurrentPageIndex();if(_80==_81){return;}
var _82=_80*_7d.getItemsPerPage();_7d.setFirstIndexOnPage(_82);_7d.onPageChanged(_80,_81);};var _83;var _84=this.getCurrentPageIndex();var _85=this.getLastPageIndex();if(_84>0){_83=createTiddlyButton(_7c,"Previous","Go to previous page (Shortcut: Alt-'<')",_7e,"prev");_83.setAttribute("page",(_84-1).toString());_83.setAttribute("accessKey","<");}
for(var i=-this.getMaxPagesInNavigation();i<this.getMaxPagesInNavigation();i++){var _87=_84+i;if(_87<0){continue;}
if(_87>_85){break;}
var _88=(i+_84+1).toString();var _89=_87==_84?"currentPage":"otherPage";_83=createTiddlyButton(_7c,_88,"Go to page %0".format([_88]),_7e,_89);_83.setAttribute("page",(_87).toString());}
if(_84<_85){_83=createTiddlyButton(_7c,"Next","Go to next page (Shortcut: Alt-'>')",_7e,"next");_83.setAttribute("page",(_84+1).toString());_83.setAttribute("accessKey",">");}}});abego.LimitedTextRenderer=function(){var _8a=40;var _8b=4;var _8c=function(_8d,_8e,_8f){var n=_8d.length;if(n==0){_8d.push({start:_8e,end:_8f});return;}
var i=0;for(;i<n;i++){var _92=_8d[i];if(_92.start<=_8f&&_8e<=_92.end){var r;var _94=i+1;for(;_94<n;_94++){r=_8d[_94];if(r.start>_8f||_8e>_92.end){break;}}
var _95=_8e;var _96=_8f;for(var j=i;j<_94;j++){r=_8d[j];_95=Math.min(_95,r.start);_96=Math.max(_96,r.end);}
_8d.splice(i,_94-i,{start:_95,end:_96});return;}
if(_92.start>_8f){break;}}
_8d.splice(i,0,{start:_8e,end:_8f});};var _98=function(_99){var _9a=0;for(var i=0;i<_99.length;i++){var _9c=_99[i];_9a+=_9c.end-_9c.start;}
return _9a;};var _9d=function(c){return(c>="a"&&c<="z")||(c>="A"&&c<="Z")||c=="_";};var _9f=function(s,_a1){if(!_9d(s[_a1])){return null;}
for(var i=_a1-1;i>=0&&_9d(s[i]);i--){}
var _a3=i+1;var n=s.length;for(i=_a1+1;i<n&&_9d(s[i]);i++){}
return{start:_a3,end:i};};var _a5=function(s,_a7,_a8){var _a9;if(_a8){_a9=_9f(s,_a7);}else{if(_a7<=0){return _a7;}
_a9=_9f(s,_a7-1);}
if(!_a9){return _a7;}
if(_a8){if(_a9.start>=_a7-_8b){return _a9.start;}
if(_a9.end<=_a7+_8b){return _a9.end;}}else{if(_a9.end<=_a7+_8b){return _a9.end;}
if(_a9.start>=_a7-_8b){return _a9.start;}}
return _a7;};var _aa=function(s,_ac){var _ad=[];if(_ac){var _ae=0;var n=s.length;var _b0=0;do{_ac.lastIndex=_ae;var _b1=_ac.exec(s);if(_b1){if(_ae<_b1.index){var t=s.substring(_ae,_b1.index);_ad.push({text:t});}
_ad.push({text:_b1[0],isMatch:true});_ae=_b1.index+_b1[0].length;}else{_ad.push({text:s.substr(_ae)});break;}}while(true);}else{_ad.push({text:s});}
return _ad;};var _b3=function(_b4){var _b5=0;for(var i=0;i<_b4.length;i++){if(_b4[i].isMatch){_b5++;}}
return _b5;};var _b7=function(s,_b9,_ba,_bb,_bc){var _bd=Math.max(Math.floor(_bc/(_bb+1)),_8a);var _be=Math.max(_bd-(_ba-_b9),0);var _bf=Math.min(Math.floor(_ba+_be/3),s.length);var _c0=Math.max(_bf-_bd,0);_c0=_a5(s,_c0,true);_bf=_a5(s,_bf,false);return{start:_c0,end:_bf};};var _c1=function(_c2,s,_c4){var _c5=[];var _c6=_b3(_c2);var pos=0;for(var i=0;i<_c2.length;i++){var t=_c2[i];var _ca=t.text;if(t.isMatch){var _cb=_b7(s,pos,pos+_ca.length,_c6,_c4);_8c(_c5,_cb.start,_cb.end);}
pos+=_ca.length;}
return _c5;};var _cc=function(s,_ce,_cf){var _d0=_cf-_98(_ce);while(_d0>0){if(_ce.length==0){_8c(_ce,0,_a5(s,_cf,false));return;}else{var _d1=_ce[0];var _d2;var _d3;if(_d1.start==0){_d2=_d1.end;if(_ce.length>1){_d3=_ce[1].start;}else{_8c(_ce,_d2,_a5(s,_d2+_d0,false));return;}}else{_d2=0;_d3=_d1.start;}
var _d4=Math.min(_d3,_d2+_d0);_8c(_ce,_d2,_d4);_d0-=(_d4-_d2);}}};var _d5=function(_d6,s,_d8,_d9,_da){if(_d9.length==0){return;}
var _db=function(_dc,s,_de,_df,_e0){var t;var _e2;var pos=0;var i=0;var _e5=0;for(;i<_de.length;i++){t=_de[i];_e2=t.text;if(_df<pos+_e2.length){_e5=_df-pos;break;}
pos+=_e2.length;}
var _e6=_e0-_df;for(;i<_de.length&&_e6>0;i++){t=_de[i];_e2=t.text.substr(_e5);_e5=0;if(_e2.length>_e6){_e2=_e2.substr(0,_e6);}
if(t.isMatch){createTiddlyElement(_dc,"span",null,"marked",_e2);}else{createTiddlyText(_dc,_e2);}
_e6-=_e2.length;}
if(_e0<s.length){abego.createEllipsis(_dc);}};if(_d9[0].start>0){abego.createEllipsis(_d6);}
var _e7=_da;for(var i=0;i<_d9.length&&_e7>0;i++){var _e9=_d9[i];var len=Math.min(_e9.end-_e9.start,_e7);_db(_d6,s,_d8,_e9.start,_e9.start+len);_e7-=len;}};this.render=function(_eb,s,_ed,_ee){if(s.length<_ed){_ed=s.length;}
var _ef=_aa(s,_ee);var _f0=_c1(_ef,s,_ed);_cc(s,_f0,_ed);_d5(_eb,s,_ef,_f0,_ed);};};(function(){function alertAndThrow(msg){alert(msg);throw msg;}
if(version.major<2||(version.major==2&&version.minor<1)){alertAndThrow("YourSearchPlugin requires TiddlyWiki 2.1 or newer.\n\nCheck the archive for YourSearch plugins\nsupporting older versions of TiddlyWiki.\n\nArchive: http://tiddlywiki.abego-software.de/archive");}
abego.YourSearch={};var _f2;var _f3;var _f4=function(_f5){_f2=_f5;};var _f6=function(){return _f2?_f2:[];};var _f7=function(){return _f2?_f2.length:0;};var _f8=4;var _f9=10;var _fa=2;var _fb=function(s,re){var m=s.match(re);return m?m.length:0;};var _ff=function(_100,_101){var _102=_101.getMarkRegExp();if(!_102){return 1;}
var _103=_100.title.match(_102);var _104=_103?_103.length:0;var _105=_fb(_100.getTags(),_102);var _106=_103?_103.join("").length:0;var _107=_100.title.length>0?_106/_100.title.length:0;var rank=_104*_f8+_105*_fa+_107*_f9+1;return rank;};var _109=function(_10a,_10b,_10c,_10d,_10e,_10f){_f3=null;var _110=_10a.reverseLookup("tags",_10f,false);try{var _111=[];if(config.options.chkSearchInTitle){_111.push("title");}
if(config.options.chkSearchInText){_111.push("text");}
if(config.options.chkSearchInTags){_111.push("tags");}
_f3=new abego.TiddlerQuery(_10b,_10c,_10d,_111,config.options.chkSearchExtendedFields);}
catch(e){return[];}
var _112=_f3.filter(_110);var _113=abego.YourSearch.getRankFunction();for(var i=0;i<_112.length;i++){var _115=_112[i];var rank=_113(_115,_f3);_115.searchRank=rank;}
if(!_10e){_10e="title";}
var _117=function(a,b){var _11a=a.searchRank-b.searchRank;if(_11a==0){if(a[_10e]==b[_10e]){return(0);}else{return(a[_10e]<b[_10e])?-1:+1;}}else{return(_11a>0)?-1:+1;}};_112.sort(_117);return _112;};var _11b=80;var _11c=50;var _11d=250;var _11e=50;var _11f=25;var _120=10;var _121="yourSearchResult";var _122="yourSearchResultItems";var _123;var _124;var _125;var _126;var _127;var _128=function(){if(version.extensions.YourSearchPlugin.styleSheetInited){return;}
version.extensions.YourSearchPlugin.styleSheetInited=true;setStylesheet(store.getTiddlerText("YourSearchStyleSheet"),"yourSearch");};var _129=function(){return _124!=null&&_124.parentNode==document.body;};var _12a=function(){if(_129()){document.body.removeChild(_124);}};var _12b=function(e){_12a();var _12d=this.getAttribute("tiddlyLink");if(_12d){var _12e=this.getAttribute("withHilite");var _12f=highlightHack;if(_12e&&_12e=="true"&&_f3){highlightHack=_f3.getMarkRegExp();}
story.displayTiddler(this,_12d);highlightHack=_12f;}
return(false);};var _130=function(){if(!_125){return;}
var root=_125;var _132=findPosX(root);var _133=findPosY(root);var _134=root.offsetHeight;var _135=_132;var _136=_133+_134;var _137=findWindowWidth();if(_137<_124.offsetWidth){_124.style.width=(_137-100)+"px";_137=findWindowWidth();}
var _138=_124.offsetWidth;if(_135+_138>_137){_135=_137-_138-30;}
if(_135<0){_135=0;}
_124.style.left=_135+"px";_124.style.top=_136+"px";_124.style.display="block";};var _139=function(){if(_124){window.scrollTo(0,ensureVisible(_124));}
if(_125){window.scrollTo(0,ensureVisible(_125));}};var _13a=function(){_130();_139();};var _13b;var _13c;var _13d=new abego.PageWiseRenderer();var _13e=function(_13f){this.itemHtml=store.getTiddlerText("YourSearchItemTemplate");if(!this.itemHtml){alertAndThrow("YourSearchItemTemplate not found");}
this.place=document.getElementById(_122);if(!this.place){this.place=createTiddlyElement(_13f,"div",_122);}};merge(_13e.prototype,{render:function(_140,_141,_142,_143){_13b=_143;_13c=_141;var item=createTiddlyElement(this.place,"div",null,"yourSearchItem");item.innerHTML=this.itemHtml;applyHtmlMacros(item,null);refreshElements(item,null);},endRendering:function(_145){_13c=null;}});var _146=function(){if(!_124||!_125){return;}
var html=store.getTiddlerText("YourSearchResultTemplate");if(!html){html="<b>Tiddler YourSearchResultTemplate not found</b>";}
_124.innerHTML=html;applyHtmlMacros(_124,null);refreshElements(_124,null);var _148=new _13e(_124);_13d.renderPage(_148);_13a();};_13d.getItemsPerPage=function(){var n=(config.options.chkPreviewText)?abego.toInt(config.options.txtItemsPerPageWithPreview,_120):abego.toInt(config.options.txtItemsPerPage,_11f);return(n>0)?n:1;};_13d.onPageChanged=function(){_146();};var _14a=function(){if(_125==null||!config.options.chkUseYourSearch){return;}
if((_125.value==_123)&&_123&&!_129()){if(_124&&(_124.parentNode!=document.body)){document.body.appendChild(_124);_13a();}else{abego.YourSearch.onShowResult(true);}}};var _14b=function(){_12a();_124=null;_123=null;};var _14c=function(self,e){while(e!=null){if(self==e){return true;}
e=e.parentNode;}
return false;};var _14f=function(e){if(e.target==_125){return;}
if(e.target==_126){return;}
if(_124&&_14c(_124,e.target)){return;}
_12a();};var _151=function(e){if(e.keyCode==27){_12a();}};addEvent(document,"click",_14f);addEvent(document,"keyup",_151);var _153=function(text,_155,_156){_123=text;_f4(_109(store,text,_155,_156,"title","excludeSearch"));abego.YourSearch.onShowResult();};var _157=function(_158,_159,_15a,_15b,_15c,_15d){_128();_123="";var _15e=null;var _15f=function(txt){if(config.options.chkUseYourSearch){_153(txt.value,config.options.chkCaseSensitiveSearch,config.options.chkRegExpSearch);}else{story.search(txt.value,config.options.chkCaseSensitiveSearch,config.options.chkRegExpSearch);}
_123=txt.value;};var _161=function(e){_15f(_125);return false;};var _163=function(e){if(!e){var e=window.event;}
_125=this;switch(e.keyCode){case 13:if(e.ctrlKey&&_127&&_129()){_127.onclick.apply(_127,[e]);}else{_15f(this);}
break;case 27:if(_129()){_12a();}else{this.value="";clearMessage();}
break;}
if(String.fromCharCode(e.keyCode)==this.accessKey||e.altKey){_14a();}
if(this.value.length<3&&_15e){clearTimeout(_15e);}
if(this.value.length>2){if(this.value!=_123){if(!config.options.chkUseYourSearch||config.options.chkSearchAsYouType){if(_15e){clearTimeout(_15e);}
var txt=this;_15e=setTimeout(function(){_15f(txt);},500);}}else{if(_15e){clearTimeout(_15e);}}}
if(this.value.length==0){_12a();}};var _166=function(e){this.select();clearMessage();_14a();};var args=_15c.parseParams("list",null,true);var _169=getFlag(args,"buttonAtRight");var _16a=getParam(args,"sizeTextbox",this.sizeTextbox);var btn;if(!_169){btn=createTiddlyButton(_158,this.label,this.prompt,_161);}
var txt=createTiddlyElement(_158,"input",null,null,null);if(_15a[0]){txt.value=_15a[0];}
txt.onkeyup=_163;txt.onfocus=_166;txt.setAttribute("size",_16a);txt.setAttribute("accessKey",this.accessKey);txt.setAttribute("autocomplete","off");if(config.browser.isSafari){txt.setAttribute("type","search");txt.setAttribute("results","5");}else{txt.setAttribute("type","text");}
if(_169){btn=createTiddlyButton(_158,this.label,this.prompt,_161);}
_125=txt;_126=btn;};var _16d=function(){_12a();var _16e=_f6();var n=_16e.length;if(n){var _170=[];for(var i=0;i<n;i++){_170.push(_16e[i].title);}
story.displayTiddlers(null,_170);}};var _172=function(_173,_174,_175,_176){invokeMacro(_173,"option",_174,_175,_176);var elem=_173.lastChild;var _178=elem.onclick;elem.onclick=function(e){var _17a=_178.apply(this,arguments);_146();return _17a;};return elem;};var _17b=function(s){var _17d=["''","{{{","}}}","//","<<<","/***","***/"];var _17e="";for(var i=0;i<_17d.length;i++){if(i!=0){_17e+="|";}
_17e+="("+_17d[i].escapeRegExp()+")";}
return s.replace(new RegExp(_17e,"mg"),"").trim();};var _180=function(){var i=_13b;return(i>=0&&i<=9)?(i<9?(i+1):0):-1;};var _182=new abego.LimitedTextRenderer();var _183=function(_184,s,_186){_182.render(_184,s,_186,_f3.getMarkRegExp());};var _187=TiddlyWiki.prototype.saveTiddler;TiddlyWiki.prototype.saveTiddler=function(_188,_189,_18a,_18b,_18c,tags,_18e){_187.apply(this,arguments);_14b();};var _18f=TiddlyWiki.prototype.removeTiddler;TiddlyWiki.prototype.removeTiddler=function(_190){_18f.apply(this,arguments);_14b();};config.macros.yourSearch={label:"yourSearch",prompt:"Gives access to the current/last YourSearch result",handler:function(_191,_192,_193,_194,_195,_196){if(_193.length==0){return;}
var name=_193[0];var func=config.macros.yourSearch.funcs[name];if(func){func(