Thursday, February 26, 2015

Better than eMail: Slack for Workgroup Communication

We're slacking off here at our non-profit organization, having discovered Slack, a cloud-based communication application that combines the functions of eMail, chat, a bit of artificial intelligence (called the Slackbot), and the ability to exchange transactions with a growing number of third-party applications including the Trello project manager. Slack solves the problem of team communication for specific topics or projects.

Let's say you are launching an e-Shop. You have the web developer, the graphic designer, the photographer, the shop manager, the back-end developer and the testers working on the project. You have calendar schedules, product photos, text copy, html and .css files all in half-a-dozen sites and places; Google Drive, Trello, your calendar, the file server, the production web site, and the sandbox web site. All this is glued together using eMails with copies to the team... each person has their own copy of the email (you hope), and relevant attachments or links to files on Google Drive, Dropbox,  your web server, or your file server. Its all a bit diffuse, and if anyone wanted to come up to speed on the whole project, then it would probably be pretty tough, because everything about the project isn't in one place.

Nine years ago, I was using Basecamp for several projects including grant applications. I have used Basecamp for many years, and sung its praises for writing grants, which is by nature a collaborative process with multiple players. About 2012, Basecamp got a major upgrade which seemed to break my workflow and processes. So, I started looking around at the alternatives, and there are bunch.

The basic unit of Slack is the team.

Teams can create channels. Channels can be for a single department, or a single project. So, for our team we created a channel for each department:
  • creative
  • development
  • admin
  • it
  • programs
Departments store their ongoing conversations within their channels. These are things that might have been communicated via eMails and attachments.  Slack can store text in a couple of structured ways; you can have a message, a snippet, or a post. A message is a simple unformatted text message similar to a chat message. (You can include emoticons). A snippet can be formatted for programming code. Finally, a post is similar to a blog post, it includes a title, and allows formatting
with fonts and bullets.  

For current projects that cross individual departments, we created specific channels.

  • eStore-Launch
  • XYZ Grant Application
  • 2015 Audit
Team members can be part of any channel, and you can invite guests who are external to the team to participate in an individual channel.  

This would all be pretty spectacular on its own, but one of the strengths of Slack is the ability to integrate with other third-party applications. We are using Slack with Trello, so that any changes made on a Trello project, get reflected in the appropriate Slack project. The integration results in what amounts to a major enhancement of both applications. 

Slack is free for basic functionality, and maybe all you ever need. Worth a look! 





Friday, February 13, 2015

Kim Klein on Attitudes Toward Money

Among the many treasures I found at our local Foundation Center Collection, was Fundraising for Social Change by Kim Klein. Now in a sixth edition, it is a 500 page book describing a full spectrum of fund-raising strategies as well as discussions of board selection, mission statements, and all of the peripheral tasks related to successful fundraising. She takes a chapter to talk about attitudes toward fundraising generally, and money in particular. 

The idea of asking for money raises another set of hindering attitudes, which are largely the inheritance of a predominantly Protestant culture infused with a Puritan ethic that affects most Americans, including those who are not Protestants. This set of values conveys a number of messages that influence our feelings and actions. For example, a Puritan ethic implies that if you are a good person and you work hard you will get what you deserve. It further implies that if you have to ask for something you are a weak person because strong people are self-sufficient. Further, most likely you have not worked hard enough and you probably don't deserve whatever you are asking for. Rounding out this series of beliefs is our deep distrust in the ability of government to solve social problems and a general convictions that the government wastes our money in unnecessary and inefficient bureaucratic red tape.

All of these beliefs can be found among people on both the left and right sides of the political spectrum as well as across age and race lines and different religious orientations. Where these beliefs will not be found is in two places:
  • Other countries. Although many countries have various taboos related to money, none have as many contradictory ones as the United States. Our taboos about talking and learning about money are not universal.
  • Children. Children have no trouble asking for money. They do not subscribe to the idea that self-sufficiency means not asking or that polite people don't ask. They ask, and the ask again and again. Our taboos about money are not natural--we are not born with them. 
Our beliefs about money are learned, and therefore they can be unlearned. The wonderful writer Ursula Le Guin once said in a lecture, "I never learned much from my teachers, but I learned a great deal from my un-teachers; the people who said to me, 'You shouldn't have learned that and you don't need to think it anymore.'"

Fundraising for social change is in part about raising the money we need, but over a longer period of time it is also about creating healthy attitudes toward money, and many people find that aspect of fundraising to be most fascinating. 

I'd love to quote the entire chapter, but that wouldn't be a blog post, it would be plagiarism! Instead, I suggest you check out the book yourself, her other books, and resources.

Thursday, January 08, 2015

Manage Linux Log Files

We were looking at log files in our various servers, with the idea that we could delete them to reclaim some disk space, but if they are properly set up with the logrotate command, they will be kept to a manageable size automatically. 

Log files live in /var/logs. There may be subdirectories within /var/logs for specific applications such as mySQL. 

Log files generally are managed through the logrotate command.  This is the program which deletes old logs, and stores and renames older versions of logs based on specifications that you put in to the logrotate.conf file.  The logrotate.conf file is typically located in /etc. It can contain defaults for all logs, and specifics for particular log files.  BUT..... 

Ugh.  Although the default specs for logs is the logrotate.conf file, some programs store their parameters elsewhere.  These include programs like apache, linuxconf, samba, cron, and syslog. 
The include parameters will read the contents of these other log parameter files, and include them in the logrotate.conf.  

(Note...these are the log configuration files,  not the log files themselves, which still appear in /var/log 

cat /etc/logrotate.d/httpd

/var/log/httpd/*log {
    missingok
    notifempty
    sharedscripts
    postrotate
        /sbin/service httpd reload > /dev/null 2>/dev/null || true
    endscript

The upshot is that most log files in /var/log will either be the current active log file for an application, or an archived version.  If the archives are compressed, then the suffix for the file will be .gz  

Reading the logs for user log-ins.  

Log files for logins, which contain the user name, time of access, and from where, are contained in two log files,  wtmp, and btmp.  On some systems there is also a utmp file. 

These are binary files, so that when accessed using cat, they will show gibberish. 
However, the last command will format them correctly.  

lastb shows the failed login attempts  (from wtmp)  - BSD
last shows successful login attempts.  (from btmp)  - BSD 
lastlog on Red Hat machines will also read the logins file. According to Wiki, this is "similar to last and lastb", but last parses a different database (wtmp and btmp). 

tail -n30  shows the last 30 lines of a log file. 
tail  by default shows the last 10 

cat /proc/version  shows the version of linux 

df -h shows the installed hard disk(s) and their useage. 

cat /etc/passwd shows all of the user accounts. 

more, less for paging.  (less allows for paging backwards) 

Sunday, December 28, 2014

Odds and Sods: Resurrection, DocBoxes

After much fiddling, I seem to have been able to get my domain techfornonprofits.com to map directly to this blog, which is hosted with bloodspot.com. For a week or two it seemed it was lost in the ether, and I'm still not exactly sure what fix finally was. But, between Google/Blogger, and my domain host at Network Solutions, it looks as if the DNS records have finally got sorted out. Techfornonprofits, the blog was started when the Blogger program was relatively new, before it was purchased by Google. My first entry was in February of 2001 which seems like ancient history now.

I'm trying to sell a few DocBoxes. These are industrial-strength Mini-Itx machines originally sourced from Logic Supply. The have AOpen cases and motherboards, using Intel Celeron chips, with 1 meg of memory, a 60 or 80 Gig hard drive, and a CD or DVD-ROM. I originally had them loaded with Windows XP embedded, or Windows 7 embedded, but have reformatted them to use Xubuntu, which is Ubuntu configured to use the XCFE interface, a lightweight front end which seems to work well with the limited 1 meg of RAM on these machines. The best thing that I liked about this was a stock installation of Xubuntu automatically found the wireless network interface, and my wireless router and my printer. With XP and Windows 7 I had to go rooting around to find drivers for both of these things.

Nothing precludes running the boxes on Windows...I've tried it with a stock Windows 7 Professional installation as well as the embedded versions. In their original lives, they were running Windows XP Embedded. The units might have a number of applications:

  • Granny or kiddie workstation 
  • Thin client
  • Industrial controller
  • Process controller
  • Mini file or media server 
  • Lightweight web server  

The picture shows the docbox with a Logitech Orbit camera on top, which was the original configuration. I'm experimenting with how best to advertise and sell these, with a couple options, Craigslist, eBay via Global Garage, a third-party seller, and Do-It-Myself eBay.

Thursday, November 13, 2014

Odds and Sods

FileMaker 13 is back at Tech Soup. A one-year subscription to FileMaker Server is $649. A full license for FileMaker Pro (the desktop client) is $194. Unfortunately, they don't offer non-profit pricing for FileMaker Pro Advanced at TechSoup, but you can inquire directly at FileMaker, where there are frequent deals. During November they are offering a 2 for 1 deal for FM Pro and FM Advanced when you buy direct.

PITA of the week: The new OSX Yosemite transmits search data by default to Apple, Microsoft, and god-knows-where. This is a reversal from previous versions of OSX. There is a fix.

Powershell is turning out to be pretty amazing. There is an entertaining introductory video series from Microsoft Virtual Academy which includes Jeffery Snover, the original PowerShell author who explains why they do things the way they do.  

Ozzie Zehner is a green-technology skeptic, in the sense that he suggests that our infatuation with alternative energy like photovoltaics and windmills really perpetuates the energy status quo. His top suggestions for committed environmentalists; empower women and girls.
Green Illusions pioneers a critique of alternative energy from an environmental perspective, arguing concerned citizens should instead focus on walkable communities, improved consumption, governance, and most notably, women’s rights.

PowerShell: Moved to new blog

PowerShell was clogging up Tech For Non-Profits. For the past several weeks I wrote a bunch of posts about using Windows PowerShell to access web APIs.  This has escalated to the extent that I thought it would worth an entire blog, so I've moved these posts to the brand-new shiny PowerShell Notebook. where I'm groping around for PowerShell mastery.  So far I have code to show how to access the following APIs:

Brightpearl
SmartyStreets
Bit.ly
Goo.gl

There are also entries about scripting FTP commands, and some general discussion of text processing. 

Take a look at PoweShellnotebook.com

Tuesday, September 16, 2014

A Database for Grant Research


I put together a grants database screen (click to view full size) to consolidate information for funding sources, and to track dates and interactions.

It is definitely an evolving project, but contains the basic information need to contact the funder, the deadline dates involved, the funder's areas of interest, and the typical range of a grant award.

So far, I've been concentrating on foundation funding. Many foundations typically ask for a letter of interest before you put together a full proposal. So, I've included multiple date fields, a deadline for a letter of interest, a deadline for a full proposal, and a date when they announce their award.

Originally I thought that this database would be mostly for research, but after working with the online grants database, Grantstation, I think I will reserve this database for funders that I really expect to submit to. Some ideas for future enhancements include:
  • Links to standard "boilerplate" paragraphs that are used in an application. 
  • Links to edit the proposal or letter directly in Word. 
  • Links to the PDFs of the proposal. 
  • Reports that create a grants calendar. 
Before anyone comments that "you should really use X software" for this purpose, I just want to say that I've used several in the past, including DonorPerfect and Blackbaud, and evaluated many others. Right now, I'm in the process of rethinking my entire workflow automation from the ground up, and this very lightweight approach is just what I'm looking for. Plus its in FileMaker, so I can run it on my Windows machines at work, or my Macs at home.     

Odds and Sods: Grantstation Membership and Training Opportunities

Grantwriting

Grantstation is a subscription-based database of grant opportunities. You can purchase a l year Grantstation Membership for $99.00 per year if you are a member of TechSoup, on September 23 and 24.

The free Grantstation 1-hour orientation webinar is well worth the time. I learned, for example that their federal grants database is a front-end using data from Grants.gov.    

Tech Soup is also sponsoring a free webinar by the CEO of Grantstation Cynthia Adams on September 18th.

Funding Rural America :
When has it ever been easy to secure funding for nonprofits and libraries in rural communities? Is there a way to level the playing field so organizations in small towns, counties, or boroughs without large metropolitan areas can compete for both government and private sector grants? What are the other options for financially supporting a rural project? Are collaborative efforts worth the effort? And, of course the biggest question: who is funding rural America?

These, and other relevant questions, along with current trends affecting rural funding will be addressed in this free, two-hour webinar presented by Cynthia Adams, CEO of GrantStation.


Finally, it is rare that we find a Grantmsanship Seminar in our neck of the woods (northern New England). But  there will be a five-day Grantmanship training seminar located in Barre VT. December 8-12, 2014

Hardware and Software

Interconnection.org is  another possible source for refurbished computers, (or a place to donate your old gear). They are one of the refurbishers for the Techsoup Refurbished Computer Initiative.

Techsoup has wonderful deals for software. A one-year subscription to FileMaker server is $209, and a subscription for FileMaker Pro (the desktop version that runs on a Mac or PC desktop) is $65.00. Visio Professional is available for $29.00.

Miscellaneous

Here is an interesting academic discussion of luck, and how to have more of it. Thanks to Jeff Duntemann for the link.

Tuesday, August 12, 2014

Rebootolator: Execute a Remote Linux Shell Script from Windows

Ok,  so, your mission, should you decide to accept it, is to restart mySQL and Apache on a remote server. This restarts a balky web site hosted by Apache, and also restarts a mySQL server which is used for a back-end for Drupal.

You want to execute this from your Windows computer.

The target computer runs CentOS 5.6 This is an (ancient) Red Hat Linux derivative, running (ancient) mySQL and Apache.

I ended up using PLink called from a Windows .CMD file to execute a bash shell script.  The shell script looks like this:

Rebootolator

#!/bin/bash -p
# Rebootolator - Reboots Apache and mySQL on a target Server
# LK Microdesign June 25, 2014
TERM="xterm"
export TERM
clear
echo
echo 'Rebooting Apache and mySQL on myServer'
echo '-----------------------------------'
echo 'Restarting mySQL'
/etc/init.d/httpd restart
echo ''
echo 'Restarting the Apache web server.'
/etc/init.d/mysqld restart
echo 'Reboot procedure completed'


Note this script is not stored on the target server, but simply put in the same folder as the windows cmd file on my windows box.  

Now for the Windows command file: 


Reboot.CMD

:: Batch file to restart services on myServer
:: Restarts mySQL and httpd 
:: Uses the Rebootolator shell script
:: LK/Microdesign August 12, 2014  
@echo off
cls
echo. 
echo.
plink -ssh username@192.168.xxx.xxx -m rebootolator.sh -pw mypass
echo.
echo.

pause >nul | echo Press any key to exit. 


So, lets deconstruct the Windows Reboot.CMD file.
The first four lines are comment lines. Turns out, you can use two colons to preface a comment in Windows, (who knew?) instead of REM.
Line 5 turns off output to the screen.
Line 6 clears the screen.
Line 7 and 8 put in blank lines.
All the work happens on line 9, using the PLINK command. PLINK is the command line version of PUTTY, a free open source terminal program for Windows workstations. Both PLINK and PUTTY are pretty wonderful and highly recommended if you need to access Linux machines from Windows.
-ssh means "use the secure socket layer protocol to log into this machine"
username@192.168.xxx.xxx is a administrator's account on the target machine,  probably the root account.
-m rebootolator.sh is the name of the shell script (above) that needs to run on the target machine.
-pw mypass is the password for the account used to log into the machine.


Deconstructing the Rebootolator.sh script:
#!/bin/bash -p  just means this is a BASH script
The two commands that actually restart the mySQL server, and the Apache server are: 
/etc/init.d/httpd restart
/etc/init.d/mysqld restart

The rest, (the echo commands) write out what  is happening at the command line. The Term command is my attempt to avoid a harmless error message that occurs when the script starts to execute.

Since I didn't realize I could host the Rebootolator.sh script in my Windows folder, I originally though I'd have to log into one Linux box, and then execute the script on the target box.  Turned out the whole thing was simpler using PLINK, which is the equivalent of SSH and SSHPASS programs used to access remote machines from the Linux command line.


Thursday, August 07, 2014

Tech Friday: FileMaker Resources

Random FileMaker-related resources: 

1. You can create an alternative icon for FileMaker 13. When you have multiple FileMaker versions installed, it sometimes is difficult to distinguish between them. FM12 and FM13 have virtually identical icons with the same color schemes. Here are alternative icons that can be installed, for both Win and Mac from HomeBase Software. HomeBase has a ton of of technical information on their web site.

2. In support of a project to integrate SmartyStreets with FileMaker, I've been doing some additional research on JSON, (Javascript Object Notation), which is a simplified version of XML.

3. Coding Standards for FileMaker

4. Modular FileMaker: is shared library of FileMaker functions. There are huge community-developed libraries for other languages such as PHP and Python. These folks are attempting a similar idea for FM. I've downloaded their JSON module, and am experimenting with it. Other examples include a nifty SQL query generator, and and another interface to Mailchimp.

Custom Functions are a way of adding small chunks of user-defined code that can be called within a FileMaker script. Brian Dunning is the guru here and curates the largest library of custom FM functions on the web. He also has sample data sets available for the U.S., Austria, Canada, and the UK. Five hundred records are free, and a million records are available for ten bucks.

5. If you need a 100,000 records or so, you could also download the database of public and private schools available at the National Center for Education Statistics. This includes demographic data as well as mailing and location addresses for schools. The data is fun to play around with. You can give yourself some sample exercises in FileMaker. For example:

6. What percentage of public school students are eligible for free or reduced lunch in your state? It is 38% in mine. Poking around in some other states, it looks like that isn't unusual; in many states it is 40-60% or more. The lunch program is often considered a proxy for the family poverty rate. But maybe that's another discussion.

 

Sync Google Apps with Microsoft OutLook 2013

I installed Microsoft Outlook 2013 the other day, along with the rest of Office 2013. I'm only about two years late, right?  Office 2013 offers little over Office 2010, but there is a somewhat cleaner design (OSX 10.7 anyone?) and really nice animations when windows open and close. Forgive me for saying this, but the interface is more Mac-like, even though it came out two years before the recent updates to OSX and IOS 7.

I originally had OutLook eMail synced to my Google Apps email. Then I wanted to also use the Outlook calendar and have changes made there reflected in my Google Apps calendar which is the group calendar that we use in the office. A quick search found the Google Outlook synchronizer, an application that sits outside of Outlook and runs interference between Outlook and Google apps.

The nice thing about this is that that it doesn't require you to put in all kinds of IMAP information into your Outlook profile...you simply give it your Google Apps eMail address and off it goes and downloads everything from Google Apps, creates a new profile within OutLook, and then uploads everything into mailboxes in Outlook (see screenshot). If it works, it will great. I includes your contact lists, shared calendars, co-workers calendars, notes, and tasks.

The poor mans's Exchange Server? 

If you use Google Apps in the office, and have Windows computers, you could have this running on each machine, and essentially use Google Apps as a back-end Exchange Server.

Monday, July 28, 2014

SmartyStreets - Validate Mailing Addresses

Some months ago I submitted a mailing list file to our mailer (we use Quad), and I was somewhat taken aback to get their report that over 5% of the addresses that we sent to them were invalid or incorrect.. I was thinking this wasn't too bad a figure, but they informed me that "people who knew what they were doing" would achieve valid rating of close to 100%. My thought was to do some "pre-validation" before sending the mail file to Quad. Turns out this can get expensive. Then I found SmartyStreets.

SmartyStreets is a web-based address validator for U.S. addresses. Using the US Postal Service official address database, SmartyStreets will validate any address that you send to it with varying amounts of correction.  Addresses are classified in a number of different ways, including:

  • Nomatch             The address is invalid.
  • Mailable              The address is valid and can receive mail. 
  • Mailable-Vacant:  The address is valid but vacant
  • Match-Inactive.   The address is valid but inactive. 
SmartyStreets has a one-off web interface which allows you to validate addresses one at a time on the fly. http://www.smartystreets.com. It will supply a nine digit zip code for valid street address, and it will suggest addresses nearby if you submit an invalid address. In short, if SmartyStreets returns an address, it will most probably be mailable. That's what I'm hoping, anyway. I just sent a new file to Quad, and we'll see what the accuracy is.

Click to Enlarge.

The web-based interface returns, not just a corrected address but additional information such as lattitude and longitude of the address, whether it is commercial or residential,and the address's time zone and congressional district. 

Smartystreets will process lists of addresses. You can simply paste in an ASCII comma-delimited list or an Excel file containing an address list, and SmartyStreets will return the list with in a mail/no-mail format, or with about 30 fields of additional information. 

SmartyStreets frequently donates their services to churches, schools, libraries, and many non-profit organizations. They have extensive documentation that explains how the validation process works, and what the results of your file can contain. And they have an API (an application programmer's interface) which will return results in XML or JSON format.  

A great addition to the mailing toolbox.  


Friday, June 13, 2014

Alabama Eye Bank runs on FileMaker Pro.

Over at  Tech for Home Healtcare,  I've described how the Alabama Eye Bank uses FileMaker Pro to manage the process of receiving donated corneas thorough finding a recipient and scheduling the surgery. It is an amazing application that shows the cross-platform versatility of FileMaker, hosted on Windows servers, and deployed to Mac workstations, iPads, and iPhones.  

Thursday, June 12, 2014

Refurbished Desktop Computers

Refurbs are for when you have more time than money. I'm not sure about the exact figure, but in many cases, I think I've ended up spending several hours per unit getting a refurbished computer back online after a hard drive failure, or just having to spend hours updating Windows and Office so that I'm confident getting the machine on the network.

We got several "really good deals" from NewEgg, for refurbished Lenovo desktop computers at $214.00. These appeared to be of "office quality", included Windows 7 Pro, and were nicely finished. Unfortunately, we have had 2/3 of the Western Digital Blue hard drives start to fail at some point. This has created no end of extra heartache for the users and an enormous amount of work for the IT staff.

NewEgg has been fine on returns, however, providing UPS shipping labels, and RMA procedures over the web.

OK....so much for NewEgg.  We're looking at alternatives.  (we have more time than money).

Techsoup has Dell refurbished computers that are prepared by a third party. For example:

Dell OptiPlex 755 Core 2 Duo Windows 7 Desktop 2.0 Ghz - 2.6 Ghz 
$286.00 
Min of 160Gb drive
Min of 2Gs RAM 
Windows 7 Professional 64 bit. 
Also includes: 
Office 2010
Adobe Flash,Reader 

One advantage here is that if you need licenses for Windows 7 and Office, they are included in the price. You would spend the $286.00 on those if you bought at retail, and maybe quite a lot less if you have a Microsoft Open, agreement. But, it like getting the hardware free.

The Dell Outlet looks promising with several machines in the $315-$390 range which still include Windows 7 as opposed to Windows 8, and have at least 500Gb drives, and 4 Gigs of RAM. These have more up-to-date processors than the Techsoup machines, and are certainly not as old. Most Dell Outlet machines were either not delivered, or were taken back within the warrenty period.

I've had solid results with Dell Outlet computers at the workstation and server level; mixed results with standard desktop machines, and a real disaster with older SX-series Optiplexes.  The best seem to be the larger ones; towers or mini-towers. Smaller machines, "mini-desktops" may have suffer from the suboptimal cooling, and the older components may have reached their design end-of-life earlier than those installed in a larger case.

One thing we have often found is that dual monitors are wonderful, and this is something that I would recommend for anyone as a matter of course. If you need an extra monitor card, these can be found from NewEgg starting at around $35.00. Best to wait until you have received the machine, because there can be variations in the slots, and the available adapter space that aren't evident from the web page.

On the Mac side, I've purchased several Macbooks, iPods, iPads, from the Apple Store. These have always worked flawlessly. The Refurb store has a 21.5 inch iMac for $1099, which is the model from September 2013. The cost is only $200 or so less that of a new, similar iMac. It includes 8 gigs of RAM, and 1 terabyte hard drive, and of course the Mavrick OS, and iWork. If you've got more money than time, and just want to get to work, this might be the way to go.

Tuesday, May 06, 2014

FileMaker: A Quick and Dirty SQL Query Screen

How to create a quick and dirty SQL screen for a FileMaker database:
1. Create a data table. I call it SQLWB. Give this table two text fields: "Query" and "Result". Both of these will be text fields.
 2. Modify the layout I call it SQL Workbench.
 3. Resize the two fields to show a larger amount of text. I add a scrollbar to the end of each field

4. Create a scripts to run the query.  I call it RunSQLQuery.

Set Field [SQLWB::Results; ExeuteSQL (SQLWB::Query;"";"")]


5. Create a script to clear the two text boxes.  I call this script ClearResults.

Set Field [SQLWB::Query;""]
Set Field [SQLWB::Results;""]

6. Add two buttons to the layout; one for each of the scripts.

The result will look something like this:












Note that this isn't a full-blown SQL implementation, it will only respond to SELECT statements. (You can't INSERT or DELETE, for example.   Also, if there any problems with your SQL syntax, the result box will only show a question mark.

Thursday, April 10, 2014

Odds and Sods

Miscellaneous Items

I've been upgrading my Dell Optiplex 620, (this of the noisy fan and the overheating case which always needs to have the top left off. ) from Windows XP to Windows 7. I used an image downloaded from the Microsoft licensing web site, and, I think this must be original version of Windows 7 that came out in 2009 or whenever, because it has applied over 150 updates, to the newly installed version, which has taken over four hours to have ready to use. Cripes, haven't they even heard of version control, or Git?

After casting around for a cloud-based web hosting company,  I'm leaning toward Linode. To start with, their documentation is terrific.

And, speaking of Git if you are trying to get your head around it, here is a great beginner tutorial that describes the theory of Git and doesn't just give another rundown of all the commands.

Visual Inspiration at Urban Sketchers.

Trello is an uncomplicated way to manage projects. There is an e-Book about Trello, Trello Dojo published by Daniel Root and published by Leanpub

As a professional nerd, occasionally I’ll run into a website, utility, or service that has potential to help the day-to-day lives of people not only the IT field, but in about any line of work. Or, perhaps even people in no line of work. Trello is just that sort of service. Their website describes it: “In one glance, Trello tells you what’s being worked on, who’s working on what, and where something is in a process.” Being free, easy to use, and constantly updated, giving Trello a try in your next project or process is a no brainer. You have almost no risk and a lot to gain.

The book comes with pointers to more than a dozen Trello templates; projects that are already set up. There is also discussion of feeding a Trello board with inputs from other sources, such as an RSS feed or eMail. An example of such an application is the Poor Man's Customer Service tracker, which takes eMails sent to your customer support web address, and automatically creates a Trello card for each issue.

Sunday, April 06, 2014

Tech Friday: API Programming 101

I've working on an add-on application for the Shopify e-Commerce web service, that allows you to have a web store completely in the cloud. What we're trying to do is to have the cart check-out function call a custom application which calculates shipping costs, because we have a slightly idiosyncratic way of dealing with those that isn't accommodated by the standard Shopify functionality.

Shopify provides an API (Application Programming Interface) to their web site which allows developers to add extra functions to a customer's web shop. Entry to the API is
provided in several different ways:
  • External Apps: Shopify calls an external function hosted at a publicly available web address. Presumably the same app provides functions for multiple customers and shops. There are bunch of these advertised on the Shopify web site. Some are free, some are supported by Shopify and some are built by third-party programmers, who make them available for $5.00 - $60.00 per month.  
  • Internal Apps: These are similar to internal apps, but are available for a single store. 
  • Embedded Apps: Haven't quite figured this out yet, but my guess is that somehow this allows an app to be embedded in a single store, and hosted within the Shopify web server, thus obviating the need for a separate web server, and public web address.
I quickly wired up our shipping calculation in PHP. Its about 12 lines of code. The problem has been to try to find how to connect it to our Shopify check-out cart. Along the way, however, I've found some interesting tools.

It is a little ironic that some of the best PHP programming tools seem to exist primarly in Windows versions, but that most of the deployment of PHP happens on Linux/Apache web servers.  Ideas:

Notepad++   The favorite of my boss, but as I said to him, "its like writing with a stick in dirt".  I use it all the time though for general editing of ASCII files.
JetBrains phpStorm I have the thirty-day trial for this, which also runs on the Mac. Looks promising.
CodeLobster for PHP Just found this free PHP editor which also has some support for javascript and CSS.

To create the publicly available web address that will host my app, I got an account with Heroku 

To examine calls to and from web services, I downloaded the the Firefox Plug-in HttpRequester.

To examine the web traffic, I use Fiddler.  To see web traffic in greater context (and to see all the data you're sending to the NSA...)  I use WireShark.

All of the above are Windows applications. Other suggestions cheerfully accepted.  (For example, on the Mac, WiresShark uses an external Xquartz graphics library, which has to be launched separately. The whole thing seems a little flakey, but it works, all the same.)

Friday, March 21, 2014

Tech Friday: XP is over, Long Live 7

With the impending demise of support for Windows XP due in April, one solution I investigated was replacing an entire workstation with a refurbished machine supplied by NewEgg. They had a several Lenovo units for sale for $219, including shipping. These included a 250 megabyte hard drive, a 2.8 or 3.0 dual-core Intel processor, and 4 Gigs of RAM, and Windows Professional 7.

We received the unit promptly, and as I unboxed it, I noticed that it appears to be an IBM-branded unit built by Lenovo of about vintage 2009. This is a little older than I would have hoped, but the Windows 7 setup seems to be going smoothly so far. The unit has a Windows "Microsoft Refurbished" sticker on it which has a valid Windows 7 code. There was also a "Made for Vista" sticker on the case, which I've scratched off. Yikes.

So far, main irritation is that the case is so small that there is no room for a full length expansion card. We normally have dual monitors for everyone, so expanding the single VGA motherboard connector may prove to be a challenge. Still, a machine with the hardware specs above, and Windows 7 for a little over $200.00... that is inexpensive. Comparable new machines were in the $600 range.

I've had mixed results with refurbs at least with Dell. A couple of Dell SX series went belly up, and one was dead on arrival. A recent Optiplex 620 series seems to overheat. I've had two full tower refurbs from Dell which have been just fine, as well as an Inspiron laptop. So, let's hope the Lenovo, er, IBM is solid.

But back to XP. Isn't it odd that although Microsoft is trying to entice users to move to the cloud, and subscription-based software, it can't seem to be in a position to offer a subscription-based support plan for $20.00 or $50.00 a year for XP. John Dvorak outlines a plan here.

I always thought Microsoft was in the business of making money. If you are a shareholder, go to the next meeting and ask the bigwigs why Microsoft is leaving what can amount to billions of dollars on the table regarding Microsoft XP. 
Microsoft is too dumb to realize that it can effortlessly monetize Windows XP using the beloved software subscription/rental scheme the company keeps discussing, but apparently has nothing but trouble trying to implement. 
I am going to describe a multi-billion dollar idea that Microsoft must consider. 
There are still approximately 500 million XP users—an estimated 29 percent of the computers in the world. Many do not want to upgrade to anything new. They are happy campers. 
These folks, according to a variety of reports, include many banks and most of the ATM networks. Smart corporate money knows that if you have something that works great for a single application you do not swap it out. You run it until something comes along that would save you enormous amounts of money. This is not happening with banking software.

Friday, March 14, 2014

ClearOS: A Linux-based Windows SBS Replacement

In my ongoing project of trying to clone a Redhat Linux server, I ran across a help file that was written for an operating system called ClearOS. I assumed that this was another Linux distribution, and ignored it at first but then, while waiting for another installation, I spent some time reading the web pages.

ClearOS is a combination of a core Linux distribution based on Red Hat and CentOS. It includes a complete set of applications to provision an entire office. Perhaps the main advantage is that it takes what are usually a number of several different disparate Linux-based programs, and it puts a slick web-based management front-end them. ClearOS is very modular; you can make things as sophisticated or simple as you want.

To get a closer look, I downloaded and installed the ClearOS Virtualbox demo. (The only glitch was a problem with the 64-bit demo; I re-downloaded the 32-bit version and that installed perfectly on Virtualbox on my iMac.)

Setup is accomplished by a wizard that walks you through a sequence of steps to install the software, connect to the internet, configure the firewall and configure additional services.

If you want to see how ClearOS looks without worrying about the installation, you can "manage" a virtual server with a Live-Demo.



The screenshot shows options for backing up local workstations.

ClearOS offers a number of different versions and support levels. You can download and run the community edition for free, a choice that I might consider to replace a Windows SBS 2011 server if there are no processes on the server that are dependent on Windows. You can install it and run it on your own dedicated hardware. (They don't recommend running the whole thing in a single virtual machine).

Or you can run it on a ClearOS hybrid appliance. These require the ClearOS Professional version which is a subscription-based support plan. The supported version can also be run on your own hardware. It includes certified and tested versions of all of the applications so that they are guaranteed to work together.

Years ago there was the Cobalt Qube, a single box which provided eMail, file and print services in a single cute box. (You can still find them on eBay). It was a great way to get an "instant network", and I was sorry to see it discontinued. The ClearOS options provide a similar instant network, and would be suitable anywhere a Windows Small Business Server might be considered.

Friday, February 28, 2014

Mining the Social Web

O'Reilly recently held a 1/2 price sale for electronic versions of some of their newer books. I bit on Mining The Social Web 2nd Edition by Matthew Russell. Having just started the book, I'm not yet in a position to really comment, but I was intrigued by several aspects. Basically the book is about using the Python programming language to access social media sites through their API (application programming interface).

Instead of having the reader set up Python on your own machine, he created a virtual environment that could be run on any platform using a program Vagrant. The VM uses a Python teaching framework called IPython Notebook which allows you to have interactive code examples embedded in text.

The combination of the virtual machine, Vagrant and IPython Notebook obviates the need to document and maintain all the possible permutations of each platform (Mac/Windows/Linux) and version of Python, all the related ancillary libraries, and whatever is needed to install and maintain updates. 

The repository for all of the Python code used in the book and the setup for the Vagrant virtual machine is on GitHub. GitHub is a cloud-based version control system, with a lightweight social media overlay that allows people to collaborate on programming projects. 

Vagrant can use different virtual machine software, but it just happens that the setup for the iPython Notebook provided uses the command-line version of VirtualBox. Vagrant installs an instance of Ubuntu 12. with all of the provisions for hosting Python and the iPython Notebook, as well as a web server which makes this available on port 8888.  So, the installation sequence for Windows is:
1. Download and install VirtualBox if you haven't already.
2. Download and install Vagrant.
3. Establish an account on GitHub
4. Download the GitHub client. This includes two applications, the GitHub terminal, and the GitHub GUI Manager.
5. Using Git, "Clone" the GitHub directory for Mining the Social Web Second Edition. This makes a copy of the GitHub directory on your local machine.
6. Start a command line session, and CD to the MTSWSE directory (whatever you've named it) Run vagrant up. This starts the creation of the virtual machine, and the full provisioning. It is about a twenty minute process. And, to be sure, given all of the output that it generates, it looks as if a multi-hour job has been automated.

A couple troubleshooting tips. If Vagrant looks like it is installing "the default VM",  or if you see a reference to PROCESS32 you've somehow missed the GitHub part... maybe the directory is wrong? What you should expect to see instead is PROCESS64.

If correct, the provisioning process will end with a line
DEBUG: Exiting. 

The real sign of success is to open a web browser and navigate to http://localhost:8888
That will bring up the main page of the IPython book..



My favorite error message to date:
DL is deprecated, use FIDDLE.  

The Vagrant documentation refers to another program called Chef. Chef manages the provisioning of servers, and is used especially in cloud,VM and multi-server installations to manage all of the server instances. 

Monday, February 24, 2014

Can't Connect: HeidiSQL + VirtualBox + mySQL + Ubuntu

After creating a new mySQL server within an Ubuntu guest OS running on VirtualBox, (how is that for a mouthful?) I wanted to connect to the server with HeidiSQL, my toolbox of choice when dealing with basic SQL operations. I expect that I would have the same problems attempting to connect with similar  client software, such as mySQL Workbench or PHPmyAdmin.

The symptoms of the problem showed up in an error message, generated by HeidiSQL:

Connect Error (2003) Can't connect to MySQL server on '192.168.219.107' (10061)  

So,  more or less in order, I did the following:
1. Create a mySQL user with an identical name and password as the Ubuntu user account. To connect to mySQL, you need to have two valid accounts:  a.) a valid Linux user account,  and b.) a valid mySQL user account. On Ubuntu, these need to be same....because you only get one chance to pass through a name and password when logging into a server from HeidiSQL. This is done by logging into the mySQL client on the Ubuntu box, and performing the following:

GRANT ALL PRIVILEGES ON *.* TO 'username'@'localhost' IDENTIFIED BY 'password' WITH GRANT OPTION;

GRANT ALL PRIVILEGES ON *.* TO 'username'@'%' IDENTIFIED BY 'password' WITH GRANT OPTION;

Exit the mySQL client, and then reload the grant tables, using the mySQL root user and password 
mysqladmin -u root -prootpassword reload 

Note that the both the user names and password are case-sensitive, and the mySQL user's name and password must be identical to the ubuntu user's name and password for Heidi to be able to log in. 

2. In VirtualBox, be sure that the network settings for your network card are for a bridged adapter. What this does is it makes the VM act like another machine on your current network. The VirtualBox default is to create its own network and subnet which are inaccessible from the host machine.  (but see note #1 below) 

3. Once you've taken care of the above items, restart the VM, and log in at the console. You need to edit the mySQL configuration file to reflect the new IP address of the VM.  Log into the VM. Usually the opening screen will show the current IP address,  in the line: 

IP address for eth0:  192.168.219.107.    

If you miss it or need to find it again, just run

ifconfig | grep "inet addr" 
This will probably show at least two lines. The first one is the current IP address.   

inet addr: 192168.219.107 Bcast:192.168.219.255 Mask:255.255.2550 
inet addr: 128.0.0.1 Mask 255.0.0.0 

Now that you know the address, you can add it to mySQL configuration file: 

sudo nano /etc/mysql/my.cnf 

Scroll down in this file until you find an entry for the bind address. This will look something like: 
bind-address = 127.0.0.1 

Change this so it it listens on the address of your VM.  I just comment out the bind-address line, and add the new line: 

#bind-address = 127.0.0.1
bind-address  = 192.168.219.107

Restart the machine.
sudo shutdown -r now 

This reloads everything, including the mySQL daemon so that it listens on the new IP address. 

That's it. You should be able to log in with Heidi now.


Notes:
1. You could probably leave the default networking setting in place, (it is NAT), and the VM will be assigned a NAT address along the lines of 10.0.2.15.  You'd still probably want to edit the mySQL file to reflect this address, and you might also want to make sure that you forward the mySQL port 3306.  I just haven't tested this, so that I can't vouch that it will work.

2. Of course, in a production situation, you'll want to choose a fixed IP address for the network address. This might involve your main router and firewall, so you may need to discuss this with the sys admins.

Anyway, now that I've actually got this thing running...I've got to get back to mySQL!

Friday, February 21, 2014

Tech Friday: Note from the Cranky Sysadmin

1. With VirtualBox chugging along, I've have developed super SysAdmin (systems administrator) powers, and created and destroyed dozens of Linux servers and desktop workstations using both Ubuntu and Red Hat operating systems. 

2. It takes over an hour to format an 8 gigabyte USB thumb drive in Windows 7. A quickformat will take just a couple of seconds. 

3. The Boss took a look at my old Linux books, (copyright 2002) and laughed a mirthless hollow laugh. "Those are really old books".  I said, "yes, they are, but they happen to describe your really old operating systems." 

4. The object of all of this virtual goodness is to upgrade the hardware on a currently production server that is around eight-years old. The problem is that the operating system, mySQL and PHP are equally old. Our idea was to transfer the existing production server, on to the new hardware, and then worry about upgrading, as we expect that some of the PHP will need to be rewritten. The problem is similar to the proverbial "change a tire on a car at 70 miles per hour". To date, I've been working on several separate scenarios:

  • Shut down the server, and do a bit-copy of the old system to the new system 
  • Leave the server running and do a copy to the new system over the network.
  • Create a new empty server with the old versions of the OS (Redhat 7) and mySQL (3.23). Dump the mySQL data off the old server and restore to the new server. 
I'm pretty confident the first option will work, as long as the server can be booted from a CD or USB, and as long as it can be rebooted once we've taken it down. 


As near as I can figure out, the server has two mirrored 40gigabyte drives, which result in a single RAID drive at /dev/hdd.  In Theory... I can do a copy of /dev/hdd to the new hardware and boot.  

[root@hive root]# fdisk -l
Disk /dev/hda: 40.0 GB, 40020664320 bytes255 heads, 63 sectors/track, 4865 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/hda1 * 1 191 1534176 fd Linux raid autodetect
/dev/hda2 192 1211 8193150 fd Linux raid autodetect
/dev/hda3 1212 1721 4096575 fd Linux raid autodetect
/dev/hda4 1722 4865 25254180 f Win95 Ext'd (LBA)
/dev/hda5 1722 1912 1534176 fd Linux raid autodetect
/dev/hda6 1913 1977 522081 fd Linux raid autodetect

Disk /dev/hdc: 40.0 GB, 40020664320 bytes
16 heads, 63 sectors/track, 77545 cylinders
Units = cylinders of 1008 * 512 = 516096 bytes
Device Boot Start End Blocks Id System
/dev/hdc1 * 1 3045 1534176 fd Linux raid autodetect
/dev/hdc2 3045 19301 8193150 fd Linux raid autodetect
/dev/hdc3 19301 27429 4096575 fd Linux raid autodetect
/dev/hdc4 27429 77536 25254180 f Win95 Ext'd (LBA)
/dev/hdc5 27429 30473 1534176 fd Linux raid autodetect
/dev/hdc6 30473 31509 522081 fd Linux raid autodetect

Disk /dev/hdd: 40.0 GB, 40020664320 bytes64 heads, 32 sectors/track, 38166 cylinders
Units = cylinders of 2048 * 512 = 1048576 bytes
Device Boot Start End Blocks Id System
/dev/hdd1 1 38166 39081968 83 Linux
The crankyness is partly of my own making, and partly, just Linux, I think. For every command there is a separate program with dozens of options making it almost impossible to memorize anything. Since everything I'm doing at this stage doesn't involve routine or repetition,it makes little sense to invest time into doing any scripting, which is, as far as I can tell, the traditional way a Linux guru copes with the command line. Usually after six months of being immersed in a foreign language, it begins to Make Sense. (that worked fine before). 

But Linux remains elusive. That's where Virtualbox comes in; you can rehearse operations and try things out.       




Wednesday, February 12, 2014

VirtualBox: Running Headless

Following up on the previous discussion of running VirtualBox from the command line, the next task is to be able to run a remote VirtualBox session, that is, a virtual machine hosted on a remote computer which may or may not be capable of a graphics display. In this case, the virtual machine is started from the host's command line using the headless parameter

VboxManage startvm RHDesktop --type headless 
This starts the vm, and automatically by default starts the remote desktop server, to allow you to log into the machine using Microsoft's Remote Desktop or any other client that uses the RDP protocol. There are at least two quirks regarding the log in with Remote Desktop:


1. When configuring the port for RDP, in VirtualBox settings,  use a port other than the default RDP 3389.
(I used 5000).

2. When connecting to the Virtual Machine using RDP, use  the the IP address of the host for the IP address.  Note that the IP address used is independent of any IP address that is assigned to the guest operating system, either as a fixed address or through NAT, when bridging.


(click on the images to see them full-sized.)

There is a good YouTube Video showing this with Windows hosts and guests.

With my RedHat guest, there seems to be issues with the mouse accurately tracking on the remote desktop.  It will be interesting to see if this is a major problem or not.

Friday, February 07, 2014

VirtualBox: Working from the command line.

Besides a superb graphical user interface, VirtualBox has a command-line interface which can be run from a command line or terminal session. You have to navigate to the installation directory for VirtualBox. On Windows 7 it is located in
C:\Program Files\Oracle\VirtualBox

Once in the directory, you control the virtual machines by using the vboxmanage command followed by a subcommand, followed by the name of the VM that you want to work with.  
Examples: 

List the currently configured VMs.  
 vboxmanage list vms

"UbServer1" {86742a7d-1188-4b48-b09a-4a0139d14930}
"UbServer2" {30bbee61-7115-4494-91fb-987a1dad74a4}
"UbDesktop" {a83379da-31e4-4064-b4c7-de4dec73195d}
"RHServer71" {bf9515d0-aead-4423-990b-3a6493712dc0}
"RHDesktop" {461c94b9-6743-4d7b-a444-6972586a562b}


This shows that I have 5 configured VMs.... two Ubuntu servers, a Ubuntu desktop, and two RedHat virtual machines.

To find out information about a specific VM:


vboxmanage showvminfo ubServer1

Name:            UbServer1
Groups:          /
Guest OS:        Ubuntu
UUID:            86742a7d-1188-4b48-b09a-4a0139d14930
Config file:     C:\Users\Larry\VirtualBox VMs\UbServer1\UbServer1.vbox
Snapshot folder: C:\Users\Larry\VirtualBox VMs\UbServer1\Snapshots
Log folder:      C:\Users\Larry\VirtualBox VMs\UbServer1\Logs
Hardware UUID:   86742a7d-1188-4b48-b09a-4a0139d14930
Memory size:     2048MB
Page Fusion:     off
VRAM size:       12MB
CPU exec cap:    100%
HPET:            off
Chipset:         piix3
Firmware:        BIOS
Number of CPUs:  1
PAE:             on
Long Mode:       off
Synthetic CPU:   off
CPUID overrides: None
Boot menu mode:  message and menu
Boot Device (1): Floppy
Boot Device (2): DVD
Boot Device (3): HardDisk 



This is the beginning of the listing, which runs about five screens. It includes all of the information shown in the GUI Settings pane, as well as additional information. 

vboxmanage startvm RHDesktop

Waiting for VM "RHDesktop" to power on...
VM "RHDesktop" has been successfully started.


This starts a named virtual machine.
The counterpart for stopping is:

vboxmanage controlvm RHDesktop poweroff

 Full manual is here



Thursday, February 06, 2014

VirtualBox: Getting Started

Having worked with virtualization software since prehistoric times... (yes, Quarterdeck DeskView and QEMM) it only has really worked for me with machines with expanded RAM. We have come a long way. The best recommendation I can give is that if you are hosting on Windows, that you would do well to have 4-8 gigs of RAM. You need to have enough RAM to comfortably run the host operating system as well as the guest operating system(s). This may be less of a problem for older guests, and maybe Linux, without a GUI (we'll see ...).

If you are expecting to be productive on the host machine when running a large guest OS, you need the RAM. For example, on my 4 gig iMac, I don't do much in the way of Mac stuff, when I'm running Windows 7 via Parallels, and I've only allocated 1.5 gigs of memory to the Windows guest... so Windows doesn't perform particularly well either.

On the other hand with my recent Windows 7 box with 8Gigs, I've been going to town hosting multiple Linux versions, as well as Windows 7 and 8 guests. In these cases I'm able to allocate 2-4 gigs to each guest.  

To get started with VirtualBox, you need to go to the website and download the version you want for your operating system. Examples here are from the Windows version. Mac and Linux versions behave similarly. Once installed and started you are shown the VB Manager.

Click to enlarge
The manager shows each virtual machine set up on the host in the left-hand pane, and then anything specific to a guest in the right hand pane. This includes all of the hardware specifications for the virtual hardware, as well as a screen shot of what is happening on the screen of the virtual machine.

The manager currently shows an instance of Red Hat Version 9 (Shrike) booting up. The preview screen is an echo of a full interactive screen which is present in its own window.  Below is an example of a Windows 7 guest running within a Windows 7 host. 



Cick to enlarge: Installing a Windows 7 guest 
Guest operating systems can be installed from physical CD/DVDs, USB drives, or disk images (.iso files). Iso files are easiest, as you don't have to worry about the VirtualBox host using external hardware. 

For example for the Windows 7 installation shown above, I assigned the Windows installation .iso file by clicking on the disk in the upper right-hand corner in the VirtualBox manager. 

Click to enlarge: Choose an disk image for installing an OS