Quantcast
Channel: news
Viewing all 74 articles
Browse latest View live

The Hub now supports reserved instances - pay up to 50% less

$
0
0

Reserve instance dialog

In response to user demand on the forum we've added Hub support for Amazon EC2 reserved instances.

Reserved instances are an Amazon EC2 feature that allows you to reserve server capacity up to 3 years in advance by paying a low one-time fee.

In exchange the hourly usage rate is significantly reduced and server capacity is guaranteed to be available when you need it.

How much will I save?

For servers that are deployed full-time you can save up to 50% of costs. For example, running a small server continuously:

Cost Unreserved Reserved (1 year) Reserved (3 years)
Up front $0 $227 $350
Hourly rate $0.085 $0.03 $0.03
Monthly cost $61 $21 $21
1 year $734
($61 x 12)
$486
($227 + $21 x 12)
$375
($375/3 + $21 x 12)
3 years $2,203 $1,460
($486 x 3)
$1,127
($375 + $21 x 12 x 3)
Total savings $0 $247/year (35%) $1,075/3 years (50%)

Similar relative cost savings are available for all server sizes.

Does reserving capacity obligate me to run a server full time?

No. At any time you can further reduce costs by shutting down unused capacity. Servers reserved for 3 years break even at 24% usage.

Which server types can I reserve?

Only EBS-backed TurnKey servers. S3-backed TurnKey servers cannot be reserved.

How exactly does it work?

Through the Hub you reserve a server instance of a specific size in a specific region. Amazon bills you a non-refundable one-time fee. The Hub applies the discounted hourly rate immediately to any matching server. You'll see this reflected in the server dashboard.

If you don't have a running server, the discounted rate will be applied to the next server you launch in the reserved size and region.

Are the reduced usage fees attached to a specific server?

You don't have to worry about that. Reserved instances are a property of the billing system, not your cloud servers. For example, if you shut down an old server and start a new one in the same region the reserved pricing will apply to the new server automatically.

For further details see Amazon's website


New Hub feature: Cloud server monitoring

$
0
0

Ladies and gentle geeks, I'm proud to announce we've just pushed out 100% free basic server monitoring to all TurnKey Hub accounts. This should make it easier to keep tabs on the health and performance of your cloud servers. Existing Hub users don't need to do anything to enjoy this new feature. It just works.

A better server dashboard

As you can see in the screenshot below, the server dashboard now includes thumbnail graphs of CPU utilization, disk IO and network traffic for the last hour:

Hub dashboard metric thumbnails

Give me more!

Alright, so instead of the last hour, you want data on how your server was doing last night? Or last week? No problem. CloudWatch samples performance at 5 minute intervals, and stores up to two weeks worth of data.

Clicking on the thumbnail graph pops up a larger interactive graph that lets you zoom in and out, sample performance metrics ondifferent timescales (e.g., hourly, daily, weekly, etc.) and move back and forward in time:

Hub detailed metrics

No installation, monitoring agents required

You don't need to install or configure any monitoring agents, because the Hub pulls statistics directly from Amazon's CloudWatch API. CloudWatch in turn gets its data directly from the virtualized hardware layer running underneath your server's operating system.

TurnKey 11.3 maintenance release - next stop Ubuntu 12.04!

$
0
0

Ho ho ho, happy holidays everyone! I know most of you are already shifting into holiday mode, so I'll keep it short and sweet.

We've just pushed out TurnKey 11.3 - the final maintenance release based on Ubuntu 10.04. The next release will be based on Ubuntu 12.04. We're already shifting into high gear for that. There will be surprises. Hopefully good ones!

Anyhow the new images we just pushed out from our CloudTask automation swarm include fixes for various bruises and scrapes, as well as the very latest security updates.

If you've already installed a previous version of TurnKey 11, you don't need to download anything because by default TurnKey is configured to automatically install all of the security updates over the network.

The maintenance release will mainly be of interest to new users and existing users doing new deployments. Especially those of you who are super impatient like mua and don't care to wait long minutes after deployment for the system to pull over a ton of security updates. This cuts down the time it takes to fully deploy Core in the cloud from 5 minutes to just 30 seconds.

Announcing TurnKey OpenVZ optimized builds (+ Proxmox VE channel)

$
0
0

OpenVZ and Proxmox VE has been a recurring topic of discussion on the forums, for which we have Jeremy to blame thank. He's done tons of research, testing, preaching, and then some.

What I love about Open Source is that if you have an itch, and the drive to scratch it yourself, you can.

That's exactly what Jeremy and Adrian did. They wanted OpenVZ optimized builds for their Proxmox VE deployments, so they developed a TKLPatch that would convert an ISO into an OpenVZ container. And if that wasn't enough, took the time to upload some of the builds to sourceforge so it would be easier for others to leverage their work.

Hats off to you guys, you rock!

TurnKey OpenVZ optimized builds

Based on Adrian's and Jeremy's work, we were able to add OpenVZ support to our build infrastructure in no time, and after some initial testing, triggered the whole appliance library to be built as optimized OpenVZ containers.

You can get them from the "Download -> More Builds" link on the appliance pages.

Pre-seeding / default passwords

Because OpenVZ builds are used in headless deployments (without a console), they include an inithook which preseeds default values and passwords (excluding the root password which is handled by the VZ CLI tools).

/usr/lib/inithooks/firstboot.d/29preseed

DB_PASS=turnkey
APP_PASS=turnkey
APP_EMAIL=admin@example.com
APP_DOMAIN=DEFAULT
HUB_APIKEY=SKIP
SEC_UPDATES=FORCE

Depending on your use case, you can preseed the values before the system is booted for the first time, or once the system has booted by executing turnkey-init.

It would be great if someone would add preseeding support to PVE... 

TurnKey Proxmox VE channel

A while back the Proxmox folks came up with the idea of adding a TurnKey channel to PVE, to allow users to download TKL appliances in the same way their custom built appliances are downloaded.

It was a great idea, but unfortunately it never got off the ground.

As I mentioned above, the great thing about Open Source is that you can scratch your own itch, and I was curious how the channel mechanism worked - so I dived in. When I came up for air I had added minimal third party channel support and a TurnKey Linux channel (github).

What this basically means is you can now download and deploy any TurnKey appliance on your PVE server in a couple of clicks without leaving your browser.

proxmox turnkey channel

I hope to see this integrated in the upcoming PVE 2.0 release [update: it's coming...]. If you're running PVE 1.9 then you can add the TurnKey channel as follows:

cd /usr/share/perl5/PVE
mv APLInfo.pm APLInfo.pm.bak
wget https://raw.github.com/turnkeylinux/pve-patches/master/PVE/APLInfo.pm

# update appliance list
pveam update

Announcing TurnKey OpenStack optimized builds

$
0
0

OpenStack Logo

As we mentioned before, making TurnKey easy to deploy on as many public and private clouds is an important goal for us. Unfortunately there are too many players in the cloud software space for us to support every single one. It's much easier to put effort into making TurnKey work well with the winning horses.

TurnKey has been supported on the leading public cloud platform Amazon EC2 from early on, not to mention simplifying management and deployment via the Hub.

OpenStack is particularly interesting, because as it is most likely the future of open source clouds.

I originally got intrigued when I heard about NASA planning to open source Nebula in 2009, which has become the basis for Nova, the compute component in OpenStack. Since then, I've been following OpenStack development from a far and have been itching to develop support for TurnKey on the platform.

The time has finally arrived, and I'm pleased to announce TurnKey optimized builds are hot out of our build farm, and available for immediate download and deployment.

You can get them from the "Download -> More Builds" link on the appliance pages.

TurnKey OpenStack optimized builds

  • EBS auto-mounting support: we've updated our custom EBSmount mechanism for OpenStack, which automatically mounts EBS devices when attached.
  • Support for automating instance setup: via the user-data scripts mechanism.
  • Automatic APT configuration on boot: saves bandwidth costs by using the closest package archive for maximum performance.
  • SSH key support: instances that are launched with a key-pair will be configured accordingly.
  • SSH host key fingerprints displayed in system log: verification of server to prevent man-in-the-middle (mitm) attacks.
  • Randomly generated root password: is set on first boot, and displayed in the system log **.
  • Randomly generated mysql/postgres passwords: the MySQL root and/or PostgreSQL postgres passwords are set to to the same random password as root **.
  • Instance metadata python library and CLI: used internally, but useful for advanced users. (learn more).

** Because OpenStack builds are used in headless deployments (without a console), they include an inithook which preseeds default values, and random passwords:

/usr/lib/inithooks/firstboot.d/29preseed

MASTERPASS=$(mcookie | cut --bytes 1-8)

cat>$INITHOOKS_CONF<<EOF
export ROOT_PASS=$MASTERPASS
export DB_PASS=$MASTERPASS
export APP_PASS=turnkey
export APP_EMAIL=admin@example.com
export APP_DOMAIN=DEFAULT
export HUB_APIKEY=SKIP
export SEC_UPDATES=FORCE

Depending on your use case, you can utilize user-data (note the security implications) to preseed during boot, or once the system has booted by executing turnkey-init.

Exemplary import of TurnKey Core on OpenStack

There are several ways of uploading an image into an OpenStack deployment, below is one way to get you started.


# cd /tmp
# tar -zxf turnkey-core-11.3-lucid-x86-openstack.tar.gz
# ls turnkey-core-11.3-lucid-x86
    turnkey-core-11.3-lucid-x86-initrd
    turnkey-core-11.3-lucid-x86-kernel
    turnkey-core-11.3-lucid-x86.img

# IMG=turnkey-core-11.3-lucid-x86

# glance add -A $GLANCE_TOKEN \
    is_public=true \
    container_format=aki \
    disk_format=aki \
    name="$IMG-kernel" \
    < /tmp/$IMG/$IMG-kernel

Added new image with ID: 5

# KERNEL_ID=5

# glance add -A $GLANCE_TOKEN \
    is_public=true \
    container_format=ami \
    disk_format=ami \
    kernel_id=$KERNEL_ID \
    name="$IMG" \
    < /tmp/$IMG/$IMG.img

Added new image with ID: 6

# glance -A $GLANCE_TOKEN index

ID  Name                                Disk Format  Container Format  Size
--  ----------------------------------  -----------  ----------------  ---------
6   turnkey-core-11.3-lucid-x86         ami          ami               688498688
5   turnkey-core-11.3-lucid-x86-kernel  aki          aki               4179712

# euca-describe-images

IMAGE   ami-00000006    turnkey-core-11.3-lucid-x86         available  public                  machine aki-00000005
IMAGE   aki-00000005    turnkey-core-11.3-lucid-x86-kernel  available  public                  kernel

Announcing TurnKey Xen optimized builds

$
0
0

Xen LogoAs we mentioned before, making TurnKey easy to deploy on as many public and private clouds is an important goal for the project.

Recently we announced TurnKey optimized builds in a number of new formats, which brings the supported list to: ISO, VMDK, OVF, Amazon EC2, OpenStack and OpenVZ.

I'm pleased to announce that we have just added Xen to the list of optimized builds. They are hot out of the build farm and available for immediate download.

You can get them from the Download link on the appliance pages.

Pre-seeding / Default passwords

The Xen images are mainly built for hosting providers who utilize the Xen Hypervisor.

Because Xen builds are used in headless deployments (without an interactive console), they include an inithook which preseeds default values and passwords.

/usr/lib/inithooks/firstboot.d/29preseed

MASTERPASS=turnkey

cat>$INITHOOKS_CONF<<EOF
export ROOT_PASS=$MASTERPASS
export DB_PASS=$MASTERPASS
export APP_PASS=$MASTERPASS
export APP_EMAIL=admin@example.com
export APP_DOMAIN=DEFAULT
export HUB_APIKEY=SKIP
export SEC_UPDATES=FORCE
EOF

You will most likely want to have your provisioning system to override the defaults by creating /etc/inithooks.conf.

Note that inithooks.conf will be blanked out once its no longer needed for security. You should also make sure that inithooks.conf includes *ALL* of the variables, otherwise the inithook system will turn on interactivity.

If you cannot support preseeding, the alternative is to have the user execute turnkey-init on first login.

Muchas Gracias to Marc from GigaTux (an official TurnKey partner) for testing the Xen images and providing feedback!
 

Rsync the entire TurnKey library from a mirror close to you in under 5 minutes!

$
0
0

Like TurnKey so much you want a local copy of all the appliances but too lazy to download individual appliance images from SourceForge by hand via browser?

I know exactly how you feel. Sloth is a virtue, and in the beginning was the command line.

So now you can use rsync or ftp to batch download the entire virtual appliance library in whatever build type you like best from a high-speed mirror near you. The way the net gods intended!

Thanks to generous donations of bandwidth and storage space from opensource friendly network samurais around the world TurnKey now has 16 high-speed rsync/ftp capable mirrors in 12 countries: China, Ireland, United Kingdom, France, Germany, Sweden, Japan, Belarus, Bulgaria, Denmark, Argentina, and Israel. And we're just getting started...

This means if your network is good enough you can now grab a copy of the entire TurnKey appliance library in any of the 6 supported build types (ISO, VMDK, OVF, Xen, OpenVZ and OpenStack) from a high-speed (e.g, 10-Gbps) mirror close to you in under 5 minutes.

A few minutes ago in a practice run I rsynced TurnKey images to my Amazon EC2 instance at an average 288 Mbps effective download rate over the network.

Wowsers! Up until I upgraded to a nice SSD a couple of weeks ago that was about as fast as I could copy files on my local hard drive.

If you find these network speeds hard to believe I invite you to log into your TurnKey Hub account and launch a small instance in Ireland. Now let's rsync the entire TurnKey appliance library in Xen format - all 8 GBs worth from the HEAnet mirror:

$ rsync rsync://ftp.heanet.ie/[snip]/turnkeylinux/

Welcome to the HEAnet mirror site, ftp.heanet.ie (http://ftp.heanet.ie/about)
-----------------------------------------------------------------------------

 NOTE: All connections and transfers are logged; if this is disagreeable,
 please disconnect now.

 * ftp.heanet.ie is located in Dublin, Ireland and operated by HEAnet, the
   Irish National Research and Education Network.

 * This is a four node cluster with 10 Gigabit access to the HEAnet backbone.

 * Please contact mirrors@heanet.ie with any operational queries.

 * You are connected to ftp-node2 (kokapetl)

-----------------------------------------------------------------------------

drwxr-xr-x         157 2012/02/12 17:05:27 .
drwxr-xr-x        4709 2012/02/08 18:00:08 iso
drwxr-xr-x        5686 2012/02/08 18:00:58 openstack
drwxr-xr-x        5434 2012/02/08 18:01:10 openvz
drwxr-xr-x        4930 2012/02/08 18:01:21 ovf
drwxr-xr-x        2759 2012/01/13 08:31:59 pve
drwxr-xr-x        5014 2012/02/08 18:00:44 vmdk
drwxr-xr-x        5266 2012/02/09 20:41:48 xen

$ time rsync -av -P rsync://ftp.heanet.ie/[snip]/turnkeylinux/xen ./

receiving incremental file list
xen/
xen/turnkey-appengine-11.3-lucid-x86-xen.tar.bz2
   299570436 100%   30.43MB/s    0:00:09 (xfer#1, to-check=83/85)
xen/turnkey-appengine-11.3-lucid-x86-xen.tar.bz2.sig
         490 100%    0.38kB/s    0:00:01 (xfer#2, to-check=82/85)
xen/turnkey-bugzilla-11.3-lucid-x86-xen.tar.bz2
   191193216 100%   34.68MB/s    0:00:05 (xfer#3, to-check=81/85)
xen/turnkey-bugzilla-11.3-lucid-x86-xen.tar.bz2.sig
         490 100%    0.48kB/s    0:00:00 (xfer#4, to-check=80/85)

[ .. snip .. ]

sent 1629 bytes  received 7993104501 bytes  36919658.80 bytes/sec
total size is 7992120707  speedup is 1.00

real    3m35.152s
user    0m29.290s
sys     0m30.220s

$

Speaking with the authority of someone that used to download shareware from a local BBS 20,000 times slower (I.e., 6MB/hour on a 14400 baud modem) I can unequivocally state that this is just friggin awesome. The future is now!

Shouts out to TurnKey mirror best buddies all over the globe:

  • Zhang from the USTC Linux User Group in China
  • James from Bytemark hosting in the UK
  • Arnoud from LIP6 in France, and Manuel from Ircam, also in France
  • Carsten from RWTH Aachen in Germany
  • Mattias from Umea Uni in Sweden
  • Mitry from Beltelecom MGTS in Belarus
  • Boian from IPACCT in Bulgaria
  • Georg from dotsrc.org in Denmark
  • Ariel from Cooperativa Telefonica in Argentina
  • Lior from the Israel Internet Association in Israel

Not yet a TurnKey mirror best buddy but thinking you might want to be? If you have the resources to provide a mirror for TurnKey in your country reach out to our global mirror commando team 24 hours a day in any real or fictional language!

TurnKey Core 12.0 RC based on Debian Squeeze

$
0
0

I'm pleased to announce a spanking brand new release candidate for TurnKey Core 12.0 - the common base for all appliances, based on the rock solid Debian Squeeze (6.0.4). The rumors were true! Hurrah! Hurrah!

This is an RC release, so take it for a spin and let us know what you think. If you come across any issues, please report them. If you have ideas on how to make it better, let us know!

Download RC:138MB ISO (changelog, signature, manifest)

Did you say Debian?

Why yes, yes I did. Here's the back story...

In mid-2010 we released our first ever Debian appliance based on Lenny. In the announcement Liraz discussed whether Debian based appliances are worth the trouble as well as some notes on Ubuntu vs. Debian.

Back then we decided not to release the entire TurnKey library based on Lenny as Squeeze was around the corner, and we were spread quite thin.

Fast forward to a few weeks ago, Liraz and I were discussing the upcoming Ubuntu LTS release, which is scheduled for April. We were deliberating when would be the best time to begin the transition.

During the conversation we revisited the idea of supporting Debian, and decided it was time. We've been wanting to support Debian since TurnKey's inception, and it seems that a significant 59% of users want Debian-based appliances "a lot"!

Rolled up my sleeves

So, I rolled up my sleeves and got to work. It wasn't too long and I had a working TurnKey bootstrap image (102MB ISO, Meta) based on Squeeze.  The most annoying part of that was dealing with the non-backwards compatible bootsplash. Turns out that was a good thing, as it forced me to do cleanup, and remove panel options that weren't actually doing anything. How nobody filed a bug on that is beyond me :)

Then I moved onto Core. Upgrading our Live Installer (di-live) was a little boring, but after fixing some bugs and seeing it work, not to mention setup LVM and install the entire OS in under a minute, it put a smile on my face.

After upgrading several key components, fixing bugs (thanks to everyone who submitted bug reports, and Jeremy for his excellent work triaging and keeping the bug tracker up to date), tweaks here and there, and testing, I was a happy camper.

To summarize, there were ups and downs but all in all it was good fun - ask my wife, I updated her on progress every evening whether she wanted to know or not.

But, there is still a long road ahead, and this is only the first milestone.

Changes

  • New and improved signature files: include detailed steps on how to verify image integrity, as well as md5 and sha1 checksums for convenience. Take a look.
     
  • Locale improvements: default locale is now set to en_US.UTF-8, updated configuration for compatibility with Squeeze. Freeing up disk space is now performed by localepurge.
     
  • Boot splash and loader: upgraded bootsplash for compatibility, removed unused panel options, and tweaked bootloader timeouts.
     
  • Live installer (di-live): upgraded for Squeeze compatibility and misc bugfixes.
     
  • Webmin: upgraded to latest upstream release and disabled inline upgrades (managed by APT).

All other changes, bugfixes and tweaks are available in the changelog.

As for the features, not much has changed except for the base distribution.

Long story short, try the RC and tell us what you think. Obviously we have immense respect for both Ubuntu and Debian, and we'd like to hear your views on where we should take it from here.

New Hub feature: Server snapshots

$
0
0

I usually get excited when adding new features to the TurnKey Hub. Recent excitement included server monitoring, reserved instances, domain management, and the Hub API.

I'm very excited about todays annoucement, not only is it awesomely useful, it's also technically cool!

So what are snapshots?

I'm sure you can guess, but let me explain anyway.

Snapshots can be used with EBS-backed instances to create point-in-time snapshots of the root filesystem, which are persisted to Amazon S3 for storage durability. Snapshots are incremental, meaning that only changes since the last snapshot are saved, taking up less storage, time, and reducing costs (see below for technical details).

Snapshots ask Amazon's fiber-optic storage backplane to save your server's disk state while it's running without impacting performance.

Ok, but what can I do with them?

Server clones

Snapshots can be used as the basis for a new server, essentially creating a clone (the cloud server equivalent of a time machine crossed with a portal to a less obnoxious alternative dimension), for example:

  • You can clone a production server to create a staging enviroment for testing new features, hacking away, whatever, without the worry of hosing your production server (guess how I tested this new feature).
  • You can essentially upgrade your servers hardware if you need the extra horse power, memory or even disk space. Say you were testing an idea with a micro instance, and now its taking off. Firstly congrats, secondly just clone the micro's latest snapshot to a larger instance size and update the DNS record / re-associate the elastic IP.
  • Let you're imagination run wild!

EBS Volumes

Snapshots can be used as a starting point for a new EBS volume, for example:

  • You mistakenly deleted a file, hosed your database, or whatever bad thing that can happen. You create a volume from the snapshot of your choice, attach it to your instance (which is auto-mounted via ebsmount) and access the data you need.
  • Again, let you're imagination run wild!

Can I schedule automatic snapshots?

You sure can! You can schedule automatic zero-load server snapshots for hourly, daily, weekly and monthly frequency, or manually create one at anytime.

There is however a snapshot limit per Amazon account, per region, so when configuring automatic scheduled snapshots, snapshot retention is also configurable to prune old snapshots, keeping you within the limit and saving you money.

Sounds cool, what does it look like?

We've added 2 new fields to the server record:

Snapshots - Server Record

And this is the snapshot dashboard:

Snapshot - Dashboard

Are there any limitations?

Snapshots only support EBS-backed instances, and not S3-backed instances. This is a technical limitation as snapshots are performed on the EBS-backed root volume, which S3-backed instance do not have.

Snapshots are saved to S3 storage, but they will not appear in your S3 buckets, nor can you access them using the standard S3 API. To access snapshot data you need to create an EBS volume or a server clone.

As mentioned above, there is a limit of the amount of snapshots each Amazon account can have, but you can request to increase your limit (specify you want the snapshots limit increased in the comments.)

Data consistency: Do not solely rely on snapshots for backups, as they may become inconsistent due to disk-buffering and locking. We use TKLBAM for our backups, and suggest you do the same.

Technical details - snapshots explained

I mentioned that snapshots are technically cool, and that they are incremental - let me try and explain what that means at how it works behind the scenes.

A snapshot of an EBS volume can be taken at anytime, which asks Amazon's fiber-optic storage backplane to save the data stored on the volume, at the block level, at that exact point-in-time, to S3 storage.

To improve performance and reduce storage space, Amazon will only copy the blocks of the volume that have changed since your last snapshot - hence incremental.

Now for the extra cool part, unlike regular incremental backup chains, you can delete any previous snapshot. Huh? What? Yep, snapshots are not chained, but are rather conceptually like a table-of-contents of pointers to saved data blocks.

When you delete a snapshot, only the data blocks that are solely used by that specific snapshot are deleted. Data blocks that are used by subsequent snapshots are not. In the below illustration, if SNAP-B is deleted, only SNAP-B:block-2 will be deleted from Amazon S3 as a newer version (SNAP-C:block-2) has already been saved.

Snapshots - Blocks


Bottom line, take snapshots for a spin and let us know what you think.

TurnKey Core 12.0 RC optimized builds

$
0
0

Last month we announced the release candidate for TurnKey Core 12.0 - the common base for all appliances, based on the rock solid Debian Squeeze (6.0.4).

It took a little longer than expected, but we've finally released all the optimized builds for TurnKey Core 12.0RC: ISO, VMDK, OVF, Amazon EC2, OpenStack, OpenVZ and Xen.

Optimized Builds

The optimized builds can be downloaded from the core appliance page, directly via the TurnKey channel in ProxmoxVE (OpenVZ), deployed in the Amazon EC2 cloud via the TurnKey Hub, or via one of the official TurnKey partners (soon).

Build specific release notes

Common (ISO)

Amazon EC2

  • Deployment: The TurnKey Hub has been updated to support Core 12.0 deployment and management (Launch new server -> 12.0). Once the full library has been updated to TKL 12.0 it will become the default, and the current release will be moved to Legacy.

VM optimized (VMDK, OVF)

  • Open-VM-Tools: Previous VM optimized builds included the proprietry VMWare-Tools, but since VMWare have released a large portion of the code under the GPL, we've moved to open-vm-tools.
  • Swap warning: VMware products might display a warning that no swap space was detected. This is a false positive, as swap is configured in LVM.

OpenStack

OpenVZ

  • Naming convention: We've updating the naming convention for openvz builds to support vanilla OpenVZ out of the box, and eliminate duplication for the Turnkey PVE channel. Thanks Jeremy!
  • Removed NTP daemon: The NTP daemon has been removed as the clock is managed by the host. Thanks Martin!
  • No more upstart hacks: Removed Ubuntu upstart hacks as they are not relevant in Debian.

Xen

  • Xen optimized kernel: Moved to the Xen optimized kernel provided by Debian (linux-image-xen-686).

As always, we need your help in testing the builds. If you come across any issues or have ideas how they can be improved, please post a comment.

Announcing TurnKey Linux 12.0: 100+ ready-to-use solutions

$
0
0

 

Ladies and gentlemen, the 12.0 release is finally out after nearly 6 months of development and just in time to celebrate TurnKey's 4th anniversary. I'm proud to announce we've more than doubled the size of the TurnKey Linux library, from 45 appliances to over 100!

As usual pushing out the release was much more work than we expected. I'd like to chalk that up to relentless optimism in the face of vulgar practical realities and basic common sense. On the flip side, we now have 100+ appliances of which 60+ are brand spanking new. Lots of good new applications are now supported.

Despite all the hard work, or maybe because of it, working on the release was the most fun I've had in a while.

You look away and then back and suddenly all this new open source stuff is out there, ready for prime time. So many innovations and competing ideas, all this free energy. We feel so privileged to have a front row seat and not just watch it all play out but also be able to play our own small role in showcasing so much high-quality open source work while making it just a bit more accessible to users.

Unlike previous releases this latest release is based on Debian, not Ubuntu. We realize this may upset hardcore Ubuntu fans but if you read on, I'll try to explain below why "defecting" to Debian was the right thing for TurnKey.

What's new?

New appliances

Upgraded appliances

Deprecated appliances

  • AppEngine: With the addition of Go, we decided to split the Google App Engine SDK appliance into 3 separate language specific appliances (appengine-go, appengine-java and appengine-python).
  • Joomla16: 1.6 has reached end-of-life, and has been replaced with the Joomla 2.5 appliance.
  • EC2SDK: Insufficient amount of interest to warrant maintenance, especially now that the TurnKey Hub exists.
  • Zimbra and OpenBravo: Will be re-introduced once TurnKey supports 64bit architectures. Sorry folks. We're still working on that.

Changes common to all TurnKey appliances - the TurnKey Core

As most of you know TurnKey Core is the base appliance on top of which all other TurnKey appliances are built and includes all the standard goodies.

Most of the ingredients Core is built out of comes directly from the Debian package repositories. Thanks to the quality of packaging refreshing these is usually very easy. A handful (e.g., webmin, shellinabox) we have to package ourselves directly from upstream because they're not in Debian. That takes more work, but still usually not a big deal.

Then there are the parts of TurnKey we developed ourselves from scratch. They're our babies and giving them the love they need usually involves more work than all the other maintenance upgrades put together. The largest of these components is TKLBAM - AKA the TurnKey Linux Backup and Migration system.

Version 1.2 of TKLBAM which went into the release received a round of much needed improvements.

The highlights:

  • Backup
    • Resume support allows you to easily recover and continue aborted backup sessions.
    • Multipart parallel S3 uploads dramatically speed up how long it takes to upload backup archives to the cloud. Dial this up high enough and you can typically fully saturate your network connection.
  • Restore
    • Added embedded squid download cache to make it easier to retry failed restores (especially large multi-GB restores!) without resorting to suicide.
    • Fixed the annoying MySQL max_packet issue that made it unnecessarily difficult to restore large tables.

In a bid to open up development to the community we've also tried to make TKLBAM more developer friendly by making its internals easier to explore via the "internals" command. We've also put the project up on GitHub. For a bit more detail, read the full the release notes for TKLAM 1.2.

TurnKey 12.0 appliances available in 7 delicious flavors

Optimized Builds

  1. ISO: Our bread and butter image format. Can be burned on a CD and run almost anywhere - including bare metal or any supported virtualization platform.
  2. Amazon EC2: 1,414 Amazon EC2 optimized builds, spanning the globe in all 7 regions, in both S3 and EBS backed flavors. The Hub has been updated and all images are available for immediate deployment.
  3. VMDK / OVF: For those of you who like a slice of virtual in your lemonade, we've got VMDK and OVF optimized builds for you too.
  4. OpenStack: The new kid on the block, OpenStack optimized builds are just a download away, get 'em while they're hot. Oh, and don't forget that these builds require a special initrd.
  5. OpenVZ : Is OpenVZ your sweet tooth? Step right up! Using Proxmox VE? The TurnKey PVE channel has been updated as well for inline download and deployment right from your web browser.
  6. Xen: And who can forget about our favorite steady rock, optimized builds for Xen based hosting providers are at your fingertips.

Want just one appliance? Follow the download links on the site to download any image in any flavor individually from SourceForge.

Want to collect them all? Thanks to the good people manning TurnKey's mirror network, you can now get all the images in a particular image format (about 20 GB) or even the whole 120 GB enchilada.

Stats:

  • 7 build formats
  • 606 downloadable virtual appliance images (120 GB worth)

Ubuntu vs Debian: the heart says Ubuntu, but the brains says Debian

One of the hardest things we had to do for this release was choose between Ubuntu and Debian. We initially planned on doing both. Then eventually we realized that would nearly double the amount of testing we had to do - a major bottleneck when you're doing a 100+ appliances.

Think about it. At this scale every one hour worth of extra testing and bugfixing translates into about 2-3 weeks of extra manwork. And to what ends? We already had way too much work to do, and the release kept getting pushed back, again and again, did it really matter whether we supported both of them like we originally intended? Would it really provide enough extra value to users to warrant the cost? They were after all quite similar...

In the end, choosing to switch over to Debian and abandon Ubuntu support for now was hard emotionally but easy on a purely rational, technical basis:

  1. Ubuntu supports less than 25% of available packages with security updates

    Remember that TurnKey is configured to auto-install security updates on a daily basis.

    This may sound dangerous to users coming from other operating systems but in practice works greats, because security updates in the Debian world are carefully back-ported to the current package version so that nothing changes besides the security fix.

    Unfortunately, if you're using Ubuntu there's a snag because Ubuntu only officially supports packages in its "main" repository section with security updates. Less than 25% of packages are included in the "main" repository section. The rest go into the unsupported "universe" and "multiverse".

    So if an Ubuntu based TurnKey appliance needs to use any of the nearly 30,000 packages that Ubuntu doesn't officially support, those packages don't get updates when security issues are revealed. That means users may get exposed to significantly more risk than with an equivalent Debian based TurnKey appliance.

    While Ubuntu are continually adding packages to its main repository, they can't keep pace with the rate of packages being added to Debian by thousands of Debian Developers. In fact, with each release the gap is growing ever wider so that an increasingly smaller percentage of packages are getting into main. For example, between Debian 6.0 (Squeeze) and the upcoming Debian 7.0 (Wheezy) release 8000 packages were added. In a comparable amount of time, between Ubuntu 10.04 (Lucid) and Ubuntu 12.04 only 1300 packages were added to main.

  2. Debian releases are much more bug-free and stable

    Stability wise, there's no comparison. Even the Ubuntu Long Term Support releases that come out every two years have serious bugs that would have held up a Debian release.

    Fundamentally this is due to the differences in priorities that drive the release process.

    Debian prioritizes technical correctness and stability. Releases are rock solid because they only happen when all of the release critical issues have been weeded out.

    By contrast, Ubuntu prioritizes the latest and greatest on a fixed schedule. It's forked off periodically from the Debian unstable version, also known as "sid" - the toy story kid that will break your toys. It then gets released on a fixed pre-determined date.

    There's no magic involved that makes Ubuntu suddenly bug-free by its intended release date compared with the version of Debian it was forked from. Ubuntu has a relatively small development team compared with Debian's thousands of developers. I think they're doing an amazing job with what they have but they can't perform miracles. That's not how software development works.

    So Debian is "done" when its done. Ubuntu is "done" when the clock runs out.

  3. Ubuntu's commercial support is not available for TurnKey anyhow

    Part of the original rational for building TurnKey on Ubuntu was Canonical's commercial support services. We theorized many higher-end business type users wouldn't be allowed to use a solution that didn't have strong commercial support backing.

    But 4 years passed, and despite talks with many very nice high level people at Canonical, nothing happened on the ground, except that one time Alon was invited to participate in an Ubuntu Developer Summit.

    We realized if we wanted to offer commercial support for TurnKey we'd have to offer it ourselves and in that case it would be easier to support Debian based solutions than Ubuntu.

Those were the main reasons for switching to Debian. Most of the people we talked to who cared about the issue one way or another preferred Debian for the same reasons, as did the majority of the 3000 people who participated in our website survey.

Don't get me wrong. Despite all the mud that has been slung at Ubuntu recently we still have a huge amount of respect for Ubuntu and even greater respect for the Ubuntu community. Sure, mistakes have been made, we're all only human after all, but let's not forget how much Ubuntu has done for the open source community.

Also, just because we don't have the resources to support Ubuntu versions of all TurnKey appliances right now doesn't mean we've ruled that out for the future. It's all about using the right tool for the job. If for some use cases (e.g., the desktop) that turns out to be Ubuntu, we'll use Ubuntu.

How to upgrade existing appliances to TurnKey 12

Simple: Just use TKLBAM to create a backup of an existing older version and then restore it on TurnKey 12.

TKLBAM only backs up files that have changed since the default installation. That means if you changed a configuration file on TurnKey 11.3 that configuration file will be backed up and re-applied when you restore to TurnKey 12.

What's next: opening up TurnKey, more developers = more appliances

To those inevitable number of you who will be disappointed that a favorite application hasn't made it into this release, please be patient, we're working within very tight resource constraints. I'll admit it's our own damn fault. We realized about a couple of years too late we haven't been open enough in the way we've been developing TurnKey.

Better late than never though. We're working very hard to fix this. Much of the work we've been doing over the last year has been to clean up and upgrade our development infrastructure so we can open it up fully to the community (and support additional architectures such as amd64!).

Soon anyone will be able to build TurnKey appliances from scratch using the same tools the core development team is using. We're working on this because it's the right thing to do as an open source project. We're also hoping it will help more people from the open source community contribute to TurnKey as equal members and lift the labor bottleneck that is preventing us from scaling from 100+ to a 1000+  TurnKey appliances.

Many, many thanks to...

  • Jeremy Davis, Adrian Moya, Basil Kurian, Rik Goldman (and his students), L.Arnold, John Carver, and Chris Musty. These guys submitted many TKLPatches that made it into this release, and even more importantly kept the community alive while we dropped off the face off the earth to focus on development.

    Their dedication and generosity have been an inspiration. We'd like the community to get to know them better so we'll soon be publishing interviews with them on the blog. Stay tuned.

  • Jeremy Davis AKA "The JedMeister": TurnKey forum moderator and all around super great guy. Jeremy is such an important part of what makes the TurnKey community tick I figured I should thank him at least twice. For emphasis. :)

  • The many rivers of upstream: Debian, Ubuntu and all of the wonderful open source communities who give love and write code for the software that goes into TurnKey.

  • Everyone else who helped test the 12.0 release candidate and provided ideas and feedback.

  • TurnKey users everywhere. Without you, TurnKey's audience, there really wouldn't be a point.

TurnKey Core 13.0 RC (i386, amd64, wheezy)

$
0
0

Drumroll please... I'm thrilled to make not one, but two announcements!

  1. TurnKey Core 13.0RC: This is a release candidate of TurnKey Core 13 based on Debian 7.0 ("Wheezy")- the upcoming version of Debian, which hasn't officially been released bu shouldn't be too far off.
     
  2. 64-bit support: TurnKey Core 13RC is available in both 32bit and 64bit versions. This means we can now guarantee that TurnKey 13 will come with 64-bit support. The wait for is nearly over. To be honest lack of 64-bit support been a nagging source of embarrassment for TurnKey for quite a while now. A significant 66% of users said this was "Very important" to them. We agree.

Download the RCs and help us test them

64bit / amd64 build146MB ISO (changelogsignaturemanifest)

32bit / x86 build: 144MB ISO (changelog, signature, manifest)

Getting ahead of the curve for TurnKey 13

In the past we've released updates to the TurnKey library a few months after a new operating system release came out. We want this to change. Ideally new releases of the TurnKey library should come out roughly in sync with new releases of the underlying Linux distribution it is based on.

We're going to try and make that happen for the next release. Hence this early version of the new TurnKey Core release candidate.

This time around there's more than just wishful thinking involved. We have better tools. Part of the motivation for improving TurnKey's development infrastructure was to simplify development and increase efficiency to make it easier for the project to keep up with new OS releases while the growth of the TurnKey library continues.

That means for the next release we won't be waiting. Even though Debian 7 ("Wheezy") is still officially in the "testing" phase, it is coming together very nicely and we're moving into position for an early release. Because we can!

Multiple architectures

As Liraz mentioned, we've been hard at work cleaning up and upgrading our development infrastructure, re-engineering the build process from the ground up, reducing complexity, adding automation and making the whole structure easily reproducible and extendible.

We've been working towards three primary goals: efficiency, multi-architecture support, and easily reproducible open development toolkit that would allow the community to participate in the development of TurnKey on equal footing.

The first two goals are now complete and we're almost there with regards to the last goal: enabling anyone to build TurnKey appliances from scratch using the same tools the core development team uses.

Getting back to the RC's - all changes, bugfixes and tweaks are available in the changelog. As for the features, not much has changed except for the base distribution at this point.

Long story short, try the RC's and tell us what you think.

TurnKey 12.1 64-bit maintenance release built with new tkldev build appliance

$
0
0

TurnKey 12.1 is out and it's the first 64-bit maintenance release to be built with tkldev - TurnKey's shiny new open appliance build system in a box.

With 64-bit support out the door, we've also pushed out a round of updates to the Hub so that users can finally deploy TurnKey on all instance sizes.

Full details on the changes to the Hub below, but first I'd like to talk a little bit about tkldev, TurnKey's new open build system. tkldev will soon be released as a standalone appliance along with the full source code to all appliances in the TurnKey Linux roster, which we will be maintaining on TurnKey's GitHub page.

tkldev: why a new build system?

As many of you know, in a bid to ease up TurnKey's dependency on the core development team, one our of strategic development goals has been to re-engineer the messy patchwork of scripts and build systems that used to be our build infrastructure with the goal of creating a self-contained "fabrication" appliance that could be used by appliance hackers to build any TurnKey appliance from source code.

We've actually had this in our sights for quite a while now. We made some bad technical decisions early on with regards to how we setup our "legacy" build infrastructure and realized a bit too late that we would have to redo everything if we wanted to get the open source community truly on board with TurnKey's development.

With everything on our plate (e.g., developing over a hundred appliances and the TurnKey Hub) it sometimes feel like we're running in place. So it's taken longer than we would have liked to make this happen. But... with a great sigh, no - heave - of relief I can proudly announce that work is finally done. Well almost. We just need to finish up the packaging and documentation.

We've been battle testing the new build system by using it to develop the upcoming TurnKey 13 release, based on Debian Wheezy, and also the TurnKey 12.1 maintenance release which I'm supposed to be announcing.

As usual, I seem to be getting a little bit ahead of myself, so let me try to get back on track.

What's new in TurnKey 12.1

64-bit (amd64) is the new default image type

Download links on the website have been replaced with 64-bit images. For users that prefer 32-bit images, they are still available for download from Sourceforge and our mirror network.

Core operating system upgrade

  • Upgraded base operating system from Debian Squeeze 6.0.5 to Debian Squeeze 6.0.7.

    Debian Squeeze is scheduled to be maintained with security updates until May 2014.

  • Upgraded all non-Debian Core components (e.g., Webmin 1.620)

  • Bugfixes and tweaks

For full details see the TurnKey Core changelog.

Fresh upstream application versions

This maintenance release includes the latest software versions for all components installed directly from upstream source code rather than the Debian package management system.

The exact details of the package version installed from upstream source code can usually be found in the appliance changelog, except where the version is determined at build time.

This includes the latest upstream versions of the main application in the following 71 appliances (from a to z):

appengine-go, appengine-java, appengine-python, appflower, b2evolution, bambooinvoice, cakephp, canvas, clipbucket, codeigniter, collabtive, concrete5, deki, django, drupal7, e107, elgg, etherpad, ezpublish, gallery, gitlab, icescrum, jenkins, joomla15, joomla25, limesurvey, magento, mambo, mibew, moodle, nodejs, omeka, openphoto, orangehrm, oscommerce, osqa, owncloud, phplist, phpnuke, phreedom, piwik, pligg, plone, prestashop, processmaker, projectpier, punbb, rails, redmine, sahana-eden, sencha, silverstripe, simpleinvoices, simplemachines, sitracker, statusnet, sugarcrm, symfony, tomatocart, tracks, twiki, typo3, ushahidi, vanilla, vtiger, web2py, wordpress, xoops, yiiframework, zencart, and zurmo.

In case you're wondering, the other 30 appliances in this release didn't get the very latest upstream version of their main application because all the major components are installed and maintained through the Debian package management system:

asp-net-apache, bugzilla, core, couchdb, dokuwiki, domain-controller, drupal6, ejabberd, fileserver, lamp, lapp, lighttpd-php-fastcgi, mahara, mantis, mediawiki, moinmoin, mongodb, movabletype, mysql, nginx-php-fastcgi, openldap, otrs, phpbb, postgresql, revision-control, roundup, tomcat, tomcat-apache, torrentserver, and trac.

The trade-off is that while the component versions may be less up-to-date, Debian provides back-ported security fixes which TurnKey automatically installs.

TurnKey headless initilization fence

This is a new feature that should make life easier for users of our headless OpenStack, Xen and OpenVZ builds.

A common problem with headless deployments of TurnKey is that turnkey-init, TurnKey's initialization wizard, can't be run on the first boot because we don't have a console that can interact with the user to properly configure the appliance (e.g., setup application and database passwords, domain name, etc.)

This typically results in frustrated users failing to understand why an uninitialized appliance doesn't seem to be working at all or complaining that they can't figure out how to log in.

The usual solution is to set a bunch of default passwords and hope users, who rarely bother (or want to bother) reading the documentation, will manage to change all of them before they get exploited.

But even when this approach sorta kinda works, having to figure out all the default passwords and rush to change them is inconvenient.

Worse, default passwords are dangerous, especially for anything connected to the Internet. They open up a window of vulnerability that can allow an attacker to compromise the system by racing to exploit your default passwords before you change them. Throw in botnets automatically scanning the network for low hanging fruit and you have a recipe for catastrophe.

TurnKey's solution to this conundrum is to leverage iptables to create a sort of virtual fence around an appliance. The fence intercepts attempts to access potentially vulnerable uninitialized applications, redirecting users instead to a mini-tutorial explaining how you need to log in as root first. On an uninitialized appliance logging in as root will automatically launch turnkey-init and help you finish setting everything up.

Introducing 64-bit images, phasing out 32-bit appliances

So why waste any time on a TurnKey 12 maintenance release (e.g., based on Debian Squeeze) when Debian Wheezy has already been released?

Basically, TurnKey 12.1 is a stepping stone for the upcoming TurnKey 13 release.

We'll be deprecating 32-bit support in TurnKey 13 and phasing it out completely by TurnKey 14. That means we'll be building all appliances in both 32-bit and 64-bit versions for TurnKey 13 but encouraging users to migrate to 64-bit because TurnKey 13 will most likely be the last major TurnKey release to come in both 32-bit and 64-bit image formats.

To make it easier for users to migrate from 32-bit to 64-bit we decided it would be a good idea to add 64-bit support to a maintenance release of TurnKey 12, the current major version of TurnKey.

That way existing users don't have to switch to a new major version of Debian while switching the operating system architecture at the same time.

Another reason we decided to bother with a maintenance release release now is that it made it easier to focus on testing the new infrastructure. Pinning down one thing that doesn't change (e.g., the base operating system + appliance roster) makes it easier to identify and work out the kinks in the pieces that have changed.

TurnKey Hub updates

  • Support for all instance sizes: with 64-bit support out the door, the Hub now supports all instance sizes, paving the way for using TurnKey for more heavy duty workloads.

  • Significant cost savings with new heavy / light reserved instances

    Up until now the Hub has only supported Medium Utilization type reservations. When we added support for reserved instances this was the only type of reservation available but Amazon have since added Light and Heavy Utilization type reservations.

    To help Hub users save significantly more money, especially when running larger instance sizes, we've added support for these types of reservations as well.

    Light Utilization reservations have a lower up-front purchase price than a Medium Utilization reservation but also a lower discount on usage fees. Heavy Utilization reservations have the highest up-front purchase price but provide greater discounts for continuously running servers.

    For example, you can save up to 76% (over $8000) of total costs when running an X-Large High-RAM instance continuously in the Amazon EC2 cloud if you purchase a Heavy Type reservation for three years.

    Purchasing a reserved instance is the Amazon EC2 equivalent of purchasing your own co-located server hardware, just with a bit more flexibility.

    The Hub provides an interactive cost savings calculator in the reserved instance dialog that shows the up-front fee, new hourly fee, break-even utilization level and expected cost savings for a continuously run server.

  • Deploy TurnKey in a cloud down under

    Some of our most important community members live in Australia so we're especially pleased to announce support for Amazon's new Sydney region.

  • New cloud servers plans & pricing

    • No hassle, free Micro support for everyone.

      You no longer have to invite anyone, sign up for a time-limited free trial, etc.. New Hub users can now immediately launch any TurnKey app as a Micro instance.

      New Amazon accounts automatically qualify for Amazon's free usage tier which provides up to 750 hours of Micro server usage each month for up to a year.

    • Budget => Bronze: The "Budget" plan has been renamed "Bronze".

      • New supported instance size: Medium (64-bit only)

        Previously only Micro, Small and Medium High-CPU instance types were supported on this plan.

      • Defaults to deploying 64-bit appliances on all instance types but allows you to choose 32-bit appliances if you want (e.g., backwards compatibility, increased memory efficiency). The upcoming TurnKey 13 release will also provide support for 32-bit appliances, but the next release after that, TurnKey 14, probably won't.

    • Hobby => Pay-per-use: The Hobby plan has been renamed Pay-per-use.

      • Supports deploying any S3-backed instance size, including all of the new large and x-large 64-bit only instance sizes.

      • Users that signed up to what we used to call the Hobby plan should receive an email from Amazon in the next few days detailing a price change which will come into effect two weeks later: we'll be dropping the markup on usage fees from 15% to 10% and adding a $10 monthly fee.

        We aren't too happy about having to change the pricing structure of what was formerly the Hobby plan. Unfortunately, in the wake of a series of steady price reductions in instance usage fees this plan has gone from bringing in a small amount of revenue, to barely covering costs, to actually costing us money. This happens because Amazon's billing system (AKA DevPay) charges us a small fixed fee per user that signs up. It used to be that the markup on usage fees covered this cost but now users with significant usage do the math and just sign up for a flat-rate plan.

        Instructions for canceling or switching to another plan are available on the Hub. See Account Details in your Hub account's EC2 account page.

    • Added three new flat-rate plans (e.g., Silver, Gold, and Platinum)

      Other than the support for larger instance sizes, the plans are similar to Bronze: a single flat-rate monthly fee allows you to deploy an unlimited number of instances with no markup on the standard Amazon EC2 usage fees.

      Current Bronze users that want to deploy larger instance sizes can upgrade seamlessly to one of the new plans by clicking "Switch Plan" on the new cloud plans and pricing page:

      https://hub.turnkeylinux.org/amazon/enable

Vote for TurnKey on SourceForge's project of the month ballot

TurnKey is a candidate for Sourceforge's June 2013 project of the month. If you like the work we're doing take a few seconds to vote for us:

http://twtpoll.com/xmrzlp

I'm no fan of demagogues and empty campaign promises, so rest assured I mean it when I promise that if you vote for us I vow, on my honor and the honor of my ancestors, to continue slaving away on TurnKey for as long as it takes to win our heroic, epic battle against the evil forces of entropy.

So vote! For TurnKey! For open source changelogs you can believe in! With your vote, together, we will create a party of superb open source software so powerful it will... WE will... repeal the oppressive laws of thermodynamics! Yes! For a better tomorrow!

Introducing TKLDev - Turnkey's appliance development and build system in a box

$
0
0

Today we proudly announce the official release of TKLDev, the new mothership of all TurnKey Linux appliance development. With TKLDev, building TurnKey Core from scratch is as easy as running make. If you can handle that you're ready to roll:

root@tkldev ~$ cd core
/turnkey/fab/products/turnkey/core
root@tkldev turnkey/core$ make

As you might expect, we're pretty stoked to finally (finally!) be able to make this announcement. It's the culmination of a couple of years worth of unglorious work behind the scenes upgrading the development plumbing that makes TurnKey possible.

I personally believe it's also a critically important milestone for TurnKey, a project I've poured my heart and soul into and would like to see grow up to embrace it's full potential.

So what's the big deal with this new build system?

Here's the problem: as much as we'd like to be the omnipresent, omnipotent, and omniscient developers that can do everything ourselves there is way more awesome open source software out there than we (or any other closed group of developers!) could ever hope to bottle up into a working library of ready-to-use solutions. Not without a great deal of help anyhow!

We started this project about five years ago. It took us a couple of years to start realizing we had a problem and another year or so to realize we had a really big problem. TurnKey was all about open source but it wasn't truly open source itself because the relationship of the core development team with the community was pretty much one sided. We could look out, but nobody could look in.

We tried plugging the hole with TKLPatch, at least as a temporary stopgap, but that didn't work and we think part of the reason was that we hadn't gone nearly far enough. It wasn't enough to make it possible for the community to customize appliances. That just made the community into limited second class citizens and while it may have been better than nothing nobody really wants to be a second class citizen in a purported open source project. Give me full access to the source code and build chain or give me death! (or... a casual hrmpf of disinterest).

We realized that meant TurnKey was either doomed to stagnate or we figure out a way to re-engineer the development process in a way that truly democratized it so that everyone could join in on the same level.

So as many of you know (we've been waving our hands around about this for a while) a couple of years ago we started working on this in earnest and we've been gradually re-engineering all our legacy build infrastructure with the ultimate goal of creating the mother of all appliances - a self-contained appliance that can be used to build any appliance (including itself) from source code.

Like Liraz said in a previous blog post:

We've actually had this in our sights for quite a while now. We made some bad technical decisions early on with regards to how we setup our legacy build infrastructure and realized a bit too late that we would have to redo everything if we wanted to get the open source community truly on board with TurnKey's development.

With everything on our plate (e.g., developing over a hundred appliances and the TurnKey Hub) it sometimes feels like we're running in place. So it's taken longer than we would have liked to make this happen.

TKLDev and appliance sources released

Long story short we've re-engineered our entire build infrastructure, bottled it up into a self-contained appliance and battle tested it by building the entire TurnKey 12.1 maintenance release. We also moved to GitHub to streamline collaboration with the community.

TKLDev is ready for immediate download in all the standard build formats in both 32-bit / i386 and 64-bit / amd64 architectures. You can also deploy it straight in the cloud via the TurnKey Hub.

Although TKLDev is relatively easy to use, it'll probably save you some time if you read through the documentation first to get an overview of how things works, development best practices, etc.

TKLDev lets you build any TurnKey appliance from scratch, directly from source. You can find the source code for all appliances in the TurnKey library at github.com/turnkeylinux-apps. All our other repositories live at github.com/turnkeylinux. We put appliances in a separate GitHub account because with so many of them they need their own namespace.

The way forward - TKL 13.0 Wheezy, a community effort?

A while back we announced release candidates for TurnKey Core 13.0, but then back tracked a little with the 12.1 maintenance release, and then side tracked again with TKLDev and the move to GitHub.

Well, it was all for a very good cause but I think we're done playing hop-scotch. With all the pieces now in place, we're moving full force on the TKL 13.0 release. The first things we'll be doing is updating all the TurnKey specific components, Core and TKLDev for 13.0. With that, the infrastructure is set to start upgrading the appliances.

We're hoping with the release of TKLDev and full appliance source code that the upcoming release will be more of a community effort. Thanks to the new infrastructure there is no special sauce, no wizard behind the curtain driving TurnKey development. We're giving Linux geeks everywhere the (super?) power to build any appliance, including Core, from scratch. What you do with it is limited only by your imagination but we're hoping at least a small part of that imagination will go towards helping us improving TurnKey for everyone's benefit.

Hopefully, we've made this stuff easy enough to use so nearly anyone with the inclination can jump in and make cool stuff happen. All the hard, really technical bits are automated so you don't need a lot of Linux experience to work with TKLDev. The same amount of experience you'd need to install and configure a regular Debian installation will take you a long way. The difference being that you can mass reproduce the results and collaborate with others on the development in ways you couldn't do otherwise.

If we've done our job well, we won't be able to anticipate all the different ways the community is going to put TKLDev to use. Some of the basic uses should be pretty obvious though. Let's say you want to improve an existing appliance. Maybe upgrade an appliance to use the latest upstream versions of your favorite open source application. Or maybe you've come up with a cool new feature that will make TurnKey Core (and by extension all other appliances) more useful? Fork, update, test, send a pull request, and your commit becomes an official part of TurnKey. It's that easy.

I can't wait to see how this evolves. Lets get hacking!

TurnKey 13 out, TKLBAM 1.4 now backup/restores any Linux system

$
0
0

This is really two separate announcements rolled into one:

  1. TurnKey 13 - codenamed "satisfaction guaranteed or your money back!"

    The new release celebrates 5 years since TurnKey's launch. It's based on the latest version of Debian (7.2) and includes 1400 ready-to-use images: 330GB worth of 100% open source, guru integrated, Linux system goodness in 7 build types that are optimized and pre-tested for nearly any deployment scenario: bare metal, virtual machines and hypervisors of all kinds, "headless" private and public cloud deployments, etc.

    New apps in this release include OpenVPN, Observium and Tendenci.

    We hope this new release reinforces the explosion in active 24x7 production deployments (37,521 servers worldwide) we've seen since the previous 12.1 release, which added 64-bit support and the ability to rebuild any system from scratch using TKLDev, our new self-contained build appliance (AKA "the mothership").

    To visualize active deployments world wide, I ran the archive.turnkeylinux.org access logs through GeoIPCity and overlaid the GPS coordinates on this Google map (view full screen):

     

  2. TKLBAM 1.4 - codenamed "give me liberty or give me death!"

    Frees TKLBAM from its shackles so it can now backup files, databases and package management state without requiring TurnKey Linux, a TurnKey Hub account or even a network connection. Having those will improve the usage experience, but the new release does its best with what you give it.

    I've created a convenience script to help you install it in a few seconds on any Debian or Ubuntu derived system:

    URL=https://raw.github.com/turnkeylinux/tklbam/master/contrib/ez-apt-install.sh
    wget -O - -q $URL | PACKAGE=tklbam /bin/bash
    

    There's nothing preventing TKLBAM from working on non Debian/Ubuntu Linux systems as well, you just need to to install from source and disable APT integration with the --skip-packages option.

    Other highlights: support for PostgreSQL, MySQL views & triggers, and a major usability rehaul designed to make it easier to understand and control how everything works. Magic can be scary in a backup tool.

    Here's a TurnKey Hub screenshot I took testing TKLBAM on various versions of Ubuntu:

    Screenshot of TurnKey Hub backups

Announcement late? Blame my problem child

As those of you following TurnKey closely may have already noticed, the website was actually updated with the TurnKey 13.0 images a few weeks ago.

I was supposed to officially announce TurnKey 13's release around the same time but got greedy and decided to wrap up TKLBAM 1.4 first and announce them together.

TKLBAM 1.4 wasn't supposed to happen. That it did is the result of a spontaneous binge of passionate development I got sucked into after realizing how close I was to making it a lot more useful to a lot more people. From the release notes:

More people would find TKLBAM useful if:

  • If it worked on other Linux distributions (e.g., Debian and Ubuntu to begin with)

  • If users understood how it worked and realized they were in control. Magic is scary in a backup tool.

  • If it worked without the TurnKey Hub or better yet without needing a network connection at all.

  • If users realized that TKLBAM works with all the usual non-cloud storage back-ends such as the local filesystem, rsync, ftp, ssh, etc.

  • If users could more easily tell when something is wrong, diagnose the problem and fix it without having to go through TKLBAM's code or internals

  • If users could mix and match different parts of TKLBAM as required (e.g., the part that identifies system changes, the part that interfaces with Duplicity to incrementally update their encrypted backup archives, etc.)

  • If users could embed TKLBAM in their existing backup solutions

  • If users realized TKLBAM allowed them to backup different things at different frequencies (e.g., the database every hour, the code every day, the system every week)

    Monolithic all-or-nothing system-level backups are not the only way to go.

  • If it could help with broken migrations (e.g., restoring a backup from TurnKey Redmine 12 to TurnKey Redmine 13)

  • If it worked more robustly, tolerated failures, and with fewer bugs

So that's why the release announcement is late and Alon is slightly pissed off but I'm hoping the end result makes up for it.

TurnKey 13: from 0.5GB to 330GB in 5 years

Big things have small beginnings. We launched TurnKey Linux five years ago in 2008 as a cool side project that took up 0.5GB on SourceForge and distributed 3 installable Live CD images of LAMP stack, Drupal and Joomla.

5 years later the project has ballooned to over 330GB spanning 1400 images: 100 apps, 7 build types, in both 64-bit and 32-bit versions. So now we're getting upset emails from SourceForge asking if the project really needs to take up so much disk space.

Yes, and sorry about that. For what it's worth, realizing TurnKey may eventually outgrow SourceForge is part of the reason we created our own independent mirror network (well, that and rsync/ftp access). Sourceforge is great, but just in case...

93,555 lines of code in 177 git repos

In terms of development, I recently collected stats on the 177 git repositories that make up the app library, self-contained build system, and a variety of custom components (e.g., TKLBAM, the TurnKey Hub).

It turns out over the years we've written about 93,555 lines of code just for TurnKey, most of it in Python and shell script. Check it out:

Late but open (and hopefully worth it)

TurnKey 13 came out a few months later than we originally planned. By now we have a pretty good handle on what it takes to push out a release so the main reason for the delay was that we kept moving the goal posts.

In a nutshell, we decided it was more important for the next major TurnKey release to be open than it was to come out early.

The main disadvantage was that Debian 7 ("Wheezy") had come out in the meantime and TurnKey 12 was based on Debian 6 ("Squeeze"). On the other hand Debian 6 would be supported for another year and since TurnKey is just Debian under the hood nothing prevented impatient users who wanted to upgrade the base operating system to Debian 7 to go through the usual automated and relatively painless Debian upgrade procedure.

So we first finished work on TKLDev, put it through the trenches with the TurnKey 12.1 maintenance release, and moved the project's development infrastructure to GitHub where all development could happen out in the open.

We hoped to see a steady increase in future open source collaboration on TurnKey's development and so far so good. I don't expect the sea to part as it takes more than just the right tools & infrastructure to really make an open source project successful. It takes community and community building takes time. TurnKey needs to win over contributors one by one.

Alon called TurnKey 13.0 "a community effort" which I think in all honesty may have been a bit premature, but we are seeing the blessed beginnings of the process in the form of a steadily growing stream of much appreciated community contributions. Not just new prototype TurnKey apps and code submissions but also more bug reports, feature requests and wiki edits.

And when word gets out on just how fun and easy it is to roll your own Linux distribution I think we'll see more of that too. Remember, with TKLDev, rolling your own Debian based Linux distribution is as easy as running make:

root@tkldev ~$ cd awesomenix
/turnkey/fab/products/turnkey/awesomenix
root@tkldev turnkey/awesomenix$ make

You don't even have to use TKLDev to build TurnKey apps or use any TurnKey packages or components. You can build anything you want!

Sadly, I've gotten into the nasty habit of prepending TKL - the TurnKey initials - to all the TurnKey related stuff I develop but under the hood the system is about as general purpose as it can get. It's also pretty well designed and easy to use, if I don't (cough) say so myself.

I'll be delighted if you use TKLDev to help us improve TurnKey but everyone is more than welcome to use it for other things as well.

3 new TurnKey apps - OpenVPN, Tendenci and Observium

  • OpenVPN: a full-featured open source SSL VPN solution that accommodates a wide range of configurations, including remote access, site-to-site VPNs, Wi-Fi security, and more.

    Matt Ayers from Amazon asked us to consider including an OpenVPN appliance in the next release and Alon blew it out of the park with the integration for this one.

    The new TurnKey OpenVPN is actually a 3 for 1 - TurnKey's setup process asks whether you want OpenVPN in client, server or gateway mode and sets things up accordingly.

    My favourite feature is the one that allows the admin to create self destructing URLs with scannable QRcodes that makes setting up client OpenVPN profiles on mobiles a breeze. That's pretty cool.

  • Tendenci: a content management system built specifically for NPOs (Non Profit Organizations).

    Upstream's Jenny Qian did such an excellent job developing the new TurnKey app that we accepted it into the library with only a few tiny modifications.

    This is the first time an upstream project has used TKLDev to roll their own TurnKey app. It would be awesome to see more of this happening and we'll be happy to aid any similar efforts in this vain any way we can.

  • Observium: a really cool autodiscovering SNMP based network monitoring platform.

    The new TurnKey app is based on a prototype developed by Eric Young, who also developed a few other prototype apps which we plan on welcoming into the library as soon as we work out the kinks. Awesome work Eric!

Special thanks

Contributing developers:

Extra special thanks

  • Alon's wife Hilla: for putting up with too many late work sessions.
  • Liraz's girlfriend Shir: for putting up with such a difficult specimen (in general).

Announcing TurnKey LXC

$
0
0

In a nutshell, LXC (LinuX Containers) can be thought of as the middle ground between a chroot on steroids and a full fledged virtual machine, making it possible to run multiple isolated "containers" on a single host.

There has been quiet a lot of interest in supporting TurnKey on LXC, so I set out to see what it would take.

Plan A, no B, ok C

The initial plan was to generate LXC optimized builds for all appliances and document how to create containers. The problem was that the process to outline in the docs wasn't very streamlined: download build, unpack build, run a couple commands, download example config, tweak config.

With plan A out the window, I moved onto plan B - create an LXC template script to automate the process. The new direction meant there was no need to generate yet another optimized build format, instead we could just patch a current build format (OpenVZ) on the fly.

This was a good plan, but lots of documentation was still required, for example: how to setup LXC, the different networking options, and the different ways to expose containers services.

Plan B was good, but code is better than documentation, moving onto plan C. Create a TurnKey LXC appliance with LXC pre-installed and configured, networking setup for both bridged and NAT, and include convenience scripts and tools for easily exposing container services.

So that's the story, and I'm pleased to announce two outcomes:

TurnKey LXC appliance

We've just released a TurnKey LXC appliance, pre-configured with everything LXC requires, including LXC itself, the TurnKey LXC template, bridge and NAT networking out of the box, dnsmasq providing DHCP and DNS services, apt-cache-ng so containers can share cached package downloads, as well as convenience scripts (nginx-proxy, iptables-nat) and related tools for exposing NAT'ed containers services to the network.

If you're new to LXC, or just want a turnkey LXC host, then this is for you. TurnKey LXC is available in all build formats for bare-metal and virtual environment installations, as well as deployment in the Amazon EC2 cloud.

Generic TurnKey LXC template

The TurnKey LXC template was developed from the get-go to be generic, and should be usable on any Linux distribution that supports LXC (which should be all of them, although we've only tested on Debian Wheezy).

Once LXC is setup on your distro, just download the template and drop it in the lxc templates directory (/usr/share/lxc/templates on Debian), and you should be able to create any of the 100+ TurnKey Linux appliances as LXC containers with a simple command:

lxc-create -n CONTAINER_NAME -t turnkey -- APPLIANCE_NAME -i /root/inithooks.conf

See the usage documentation for further details.


Keep in mind, this is the initial release of TurnKey LXC, so let us know what you think. If you have ideas for improvement, or if you come across any issues (ie. we haven't tested all appliances as of yet), drop us a line.

As with all TurnKey appliances, the source code is available on GitHub and we love pull requests.

Announcing TurnKey Docker optimized builds

$
0
0

docker logoAs we've mentioned before, making TurnKey easy to deploy no matter your platform of choice is an important goal for the project. TurnKey already supports a mirade of build types including ISO, VMDK, OVF, Amazon EC2, OpenStack, OpenVZ, OpenNode, Xen, and recently added support for LXC.

I'm pleased to announce that today we are updating the above list, with support for Docker.

 

Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.

Deploying TurnKey on Docker

All TurnKey appliances are available on the public docker index (generously provided by Docker, Inc), which streamlines deployment. For example:

docker pull turnkeylinux/core-13.0
docker run -i -t -d turnkeylinux/core-13.0

Docker containers can be run in the foreground or the background, so we've tried our best to support all use cases with regards to initialization (aka. inithooks) - secret regeneration, setting of passwords, application configuration, etc. 

Depending on your use case, we recommend two options:

Option 1: Initialization via ssh (interactive)

On first login, you will be prompted to initialize the appliance.

CID=$(docker run -i -t -d turnkeylinux/core-13.0)
CIP=$(docker inspect -format='{{.NetworkSettings.IPAddress}}' $CID)
docker logs $CID | grep "Random initial root password"
ssh root@$CIP

Option 2: Create new image with preseeded values (non-interactive)

The appliance will initialize itself with the provided configuration.  Once initialized, the configuration will be deleted. For more information see inithooks.

mkdir /root/wordpress

cat > /root/wordpress/inithooks.conf <<EOF
export ROOT_PASS=secretrootpass
export DB_PASS=secretmysqlpass
export APP_PASS=secretadminwppass
export APP_EMAIL=admin@example.com
export APP_DOMAIN=www.example.com
export HUB_APIKEY=SKIP
export SEC_UPDATES=FORCE
EOF

cat > /root/wordpress/Dockerfile <<EOF
FROM turnkeylinux/wordpress-13.0
ADD inithooks.conf /etc/inithooks.conf
EOF

docker build -t wordpress-13.0 /root/wordpress
docker run -i -t -d wordpress-13.0

Notes

Pre-configured run command

Docker is designed for "application or process" containers - for example, running mysql, and only mysql. Docker short-circuits /sbin/init so you can't really "boot" a container like in vanilla LXC.

To work around this, we've included /usr/sbin/start.sh (default run command) which will start all services and drop to a shell. When the shell is exited, the services will be stopped. For this reason, SSH is recommended for regular console usage.

STDIN and TTY options required

The -i and -t options are required to attach STDIN and allocate a TTY to the container. Unfortunately this cannot be pre-configured as it is not yet supported.

Pre-configured to expose ports

All TurnKey Docker appliances are configured to expose their custom services. This means that the host can access the services, but they are not exposed to the network.

Exposing ports to the network needs to be done at runtime (docs), for example:

# bind port 80 on the host to the container's port 80
docker run -i -t -d -p 80:80 turnkeylinux/lamp-13.0

Skipping security updates on first boot

During development I added support to start.sh to override the default SEC_UPDATES value to speed up my testing. I was going to remove this support or leave it undocumented, but decided others might find it useful when testing (and only in testing).

# THIS IS NOT RECOMMENDED, USE AT YOUR OWN RISK!
docker run -i -t -d -e SEC_UPDATES=SKIP turnkeylinux/openldap-13.0


Keep in mind, this is the initial release of TurnKey Docker support, so let us know what you think. If you have ideas for improvement, or if you come across any issues (ie. we haven't tested all appliances as of yet), drop us a line.

And then there were three...

$
0
0

Hi all! This is my virgin TurnKey blog post. Many of you on the forums would have come across me in your travels no doubt. I have been a volunteer serial poster on the forums now for many years. I have even had the privilege of having a blog post written about me by Liraz (one of the core TurnKey devs).

jeremyThose that pay attention to the 'karma' stats (see bottom of the forums page) may have noticed that whilst I still retain the "Most awesome of all time" crown I'm not in the "Most awesome this month" list, despite my posting frequency still being quite high. Forum regulars may have also noticed a subtle change in the way that I have been writing my posts lately. I feel like I write with a little more authority now. And that is for a very good reason!

Earlier this year I had the incredible honour of being asked by Alon and Liraz to join the TurnKey team on a full time basis! I have felt like part of the team for quite some time now, but to be formally asked to quit my job and come on board with TKL full-time blew my mind! But it also scared me a little bit too!

Never the less, I sucked it up and gave notice at my old job and started work on TurnKey part time almost straight away (I had a few lose ends to tidy up before leaving my old job). So here I am, the latest TurnKey Linux developer (in the loosest sense of the term); joining forces with the dynamic duo that is Alon and Liraz!

So what does it all mean for TurnKey?

So what will this mean for TurnKey Linux? Well from the outside, probably not a lot really. I will still be posting on the forums and on GitHub but I will be doing other (more behind the scenes) stuff too! :D

One of the most important roles will be to talk with people from other organisations, like hosting providers (to encourage them to partner with TurnKey Linux and host TurnKey appliances on their cloud/VPS platform) and to start some of the discussions with upstream developers; especially Debian (to seek to contribute TKL code back upstream and develop ideas such as upstream usage of TKLDev).

I will also be taking up the 'level one support' role. So chances are if you contact TKL Support email it will probably be me politely suggesting that you post in our forums :D (unless of course it is the sort of question that should be handled via email...).

Also I will be pondering questions about the community (with Alon and Liraz) regarding such things as how to build a better, bigger but sustainable TurnKey library, if and how to build in financialrewards and other such exciting ideas!

Most of all though, my role will be to make sure that any important TKL business (whether from community members, hosting providers, upstream developers, general users or anyone else really) gets taken care of and that Alon and Liraz are up to speed with what is happening without needing to take too much time out from development. Hopefully this will mean that the core devs do more core deving but also issues that I don't know how to answer can get their attention quicker!

So let the fun begin...! :D

TurnKey v14.0 RC1 is LIVE! (aka we need YOU!)

$
0
0

It is with great pleasure that I announce the release of Core v14.0RC1 and TKLDev v14.0RC1! But first a little history and context... We had always intended to do a v13.1 maintenance release. Ideally it should have been out long ago! Actually we probably should have released v13.2 or perhaps even v13.3 by now, but time just got away from us. :(

Things have been pretty crazy behind the scenes here over the last year or so. Alon and Liraz have both been tied up with other important stuff; including major improvements to both the Hub and TKLBAM. So with the recent release of Debian Jessie we have decided to cut our losses and sidestep the v13.1 maintenance release. As Alon and Liraz are still pretty busy, I am driving the v14.0 release. Alon is providing assistance and guidance but I'm pretty much running it, with a few freelance dev guys doing a lot of the work. :)

Core and TKLDev

In times passed, our first release candidate (RC) only included Core but this time around we thought it would make much more sense to release TKLDev at the same time! This means that the community (yes I'm looking at YOU!) can not only test Core - the base appliance that all others are built on. But you can also test the build, install and usage of the whole library using TKLDev! How cool is that?!

Update: Here's a tutorial showing step by step instructions for building any TurnKey ISO from source.

Download the RCs and help us test them

Core (64bit / amd64 build)205MB ISO  ( changelogsignaturemanifest )

TKLDev (64bit / amd64 build)263MB ISO  ( changelogsignaturemanifest )

Help needed to complete the upgrade to Jessie

As many seasoned TurnKey users would know, historically a major version bump means a rebase on a new OS version. And as I mentioned above, v14.0 is no exception. TurnKey Linux v14.0 is built on top of Debian Jessie (aka Debian 8).

As there are significant changes that come with Jessie we could really do with some community help on this one. Alon has built these first RCs but I am sure that there will be plenty of issues we'll need to resolve. Just finding the bugs and reporting them on our Issue Tracker (or even just posting here) would be a massive help!

64 bits...

Many may note that there are only 64 bit downloads available this time around. Some may recall that we considered discontinuing 32 bit support back with the v13 release. Even though we decided not to, we have considered the 32 bit builds to be legacy builds for some time now. For this release we are focused on the 64 bit release only at this stage.

We are not totally clear whether this will be the end of 32 bit builds or not. We are deferring that decision for now and will revisit it once we get closer to a final stable release. As a general rule though, the only people that should be affected are those using really old hardware.

Track the progress

[UPDATE] the table linked to below may be out of date. Please see this google spreadsheet for up to date info regarding the v14.0 appliance updates.

As with the v13.0 development we plan to use the GitHub based dev wiki (v14.0 page - appliance status table) and issue tracker so everyone can keep up to speed on progress. Alon and/or Liraz will be merging pull requests at least once per week (often daily). Hopefully we'll get any show-stopper bugs sorted out pretty quickly.

So with a bit of community involvement tweaking and upgrading I hope v14.0 to be our best release yet! I know that with your help we can make it happen! :)

New appliances

There are also a number of new appliances that have been community developed that we hope to get into this release. If you have a favourite open source project or are associated with open source development and would like to see it released as a new TurnKey appliance then now is the time to act! If your appliance is ready and meets the standards by the time the stable release is ready then we'll include it in v14.0!

Marching towards v14.0 RC2

$
0
0

As anyone who has been paying attention would realise; v14.0RC1 has been out for over a month now. And here at TurnKey central we have been very busy updating the appliances. To be honest I wish that I was announcing the final release, but alas we're not there yet - there's still lots of work to do...

But we are certainly getting closer and I hope to be announcing a RC2 release very soon... Hopefully we may even be able to provide some prebuilt ISOs of v14.0RC2 appliances too! Obviously you could build them yourself but hey, we get that it's easier if that part is already done...

Progress Report

As you can see on our v14.0 Appliance Build Status Spreadsheet many of them (70+) are basically complete. While we've been at it we have managed to close 30 issues (mostly bugs). However we still have over 100 to go....

So most of the existing appliances are done. But we still have to test most of them and we also have a ton of bugs to squash. Depending on how things go we may need to hop over some of the feature requests to v14.1...

Outstanding appliances

Of particular note are the appliances which are still outstanding. These are (with some notes): [updated 07-07-2015]

  • asp-net-apache - apache-mod-mono didn't make it into Jessie so will need to use upstream
  • django - TKL Django example app needs rewriting from scratch
  • domain-controller - needs testing (and possibly tweaking) due to Samba4 now being default Samba
  • drupal7 - upgrade existing drupal6 appliance (so that it uses drupal7 from Debian repos - might rename 'drupal-stable') Note: we decided to just reuse the existing Drupal7 appliance (installed from upstream with drush)
  • drupal8 - update of existing drupal7 appliance (install from upstream - probably rename drupal-dev) Thinking perhaps just name drupal8.
  • ejabberd - Speeqe not compatible with Django1.7; either need to install replacement app (such as CandyChat?) or install Django1.4 to be deferred for now
  • fileserver - Ajaxplorer has been radically updated since v13.0 and is now Pydio - old integration doesn't work... - we ended up doing things a little differently this time...
  • joomla25 - needs update to latest stable Joomla (and renaming) - now Joomla3 (tracking the v3.x.x branch)
  • lighttpd-php-fastcgi
  • mysql - currently being worked on by Ken
  • nginx-php-fastcgi
  • openldap
  • owncloud - oC7 in Debian repos; but probably need to document installation/upgrade from upstream... - partially done
  • postgresql - also currently being worked on by Ken
  • torrentserver - relies on fileserver to be deferred for now
  • trac - some packages missing from Jessie
  • typo3 - currently in progress
  • xoops
  • lxc
  • Keep in mind that some of these are already in progress but if you have some specific interest (e.g. you use that appliance, are involved with upstream, really love that software, are an expert, etc) then please post (or contact me directly) and I can give you some guidance.

    Also as noted above we still have plenty of issues to close, so perhaps you can help with those?

    Credit Where Credit is due!

    I would also like to take this opportunity to give some credit to those that have been working to get this release to where it is. Some are long standing TKL community members who have contributed much over a number of years; others are new comers who have just started helping us out.

    Some have contributed new appliances, or updated existing appliance build code. Some have reported bugs and some have fixed them I have tried to name everybody but I'm sure that I've missed some. Please feel free to speak up if you didn't get a mention and think that you should!

    In no particular order:

  • Jonathan Struebel
  • Jeroen Peters
  • Landis Arnold (TKL)
  • Ken Robinson (TKL)
  • Tim Hibberd (TKL)
  • John Carver (TKL)
  • @SiKing
  • Lloyd Ernst and Cloudstaff.com including Bryan Santos, Adrian Del Rosario, Rodolfo Lansangan & Justin Ashley Salvacion
  • Tim Buckland
  • Hans Harder
  • @wvengen
  • Anton Pyrogovskyi
  • Stefan Davis
  • Alfonso Valdes
  • Vinitha Cejo John
  • Mark Krueg
  • Richard van Dijk
  • Rob Fantini
  • pee

    Part of me would really like to get into the detail of what each of these great people have contributed but I'd probably never finish this blog post. Plus it would make it more likely that I would miss some important info...

    More Help Needed

    Despite the progress and all the great work already done by those noted above (and others I've possibly missed) we still need a big push to get this over the finish line. I am concerned that without some more hands pushing us along I might be writing a similar post in another month's time. Please help me avoid that! :)

Viewing all 74 articles
Browse latest View live