Friday, January 31, 2014

19 Rails Tricks Most Rails Coders Don’t Know

When looking at my own Rails code and that of the community as a whole, I often see places where certain Rails techniques could have been used, but weren't. As much for my own memory as yours, I thought I'd list down some Rails tricks and tips that can make your application or code more efficient:
Benchmark logic in your controller actions - It's really easy. Use the benchmark class method available on all your models like this:
User.benchmark("adding and deleting 1000 users") do
 1000.times do
  User.create(:name => 'something')
  x = User.find_by_name('something')
  x.destroy
 end
end
Of course, your code would be a lot better ;-) The regular SQL logs are not shown when within the benchmark sections. Only the benchmark results are shown.
Nested-Set

acts_as_nested_set
- Almost everyone is familiar with acts_as_tree, but acts_as_nested_set snuck into Rails quietly. It's much like acts_as_tree, but with the added benefit that you can select all of the children (and their own descendants) of a node with a single query. A list of the instance methods is available.
Easier collections with to_proc - Sick of writing things like post.collect { |p| p.title } or post.select { |p| p.activated? }.collect{ |p| p.title} ? A little Ruby hackery that allows you to convert symbols into proc references makes it easy. You can write post.collect(&:title) or post.select(&:activated?).collect(&:title) instead! Learn a lot more about this.
Convert arrays to sentences in views - If you were collecting a bunch of names to be shown in a view, you might end up with an array like ['Peter', 'Fred', 'Chris'], and joining these with commas and inserting 'and' before the final one is a common pain. Not so, if you use the array method to_sentence as provided in Rails. names.to_sentence would return Peter, Fred, and Chris.
Send files back to the user - Usually, static files can be retrieved by using the direct URL and circumventing your Rails application. In some situations, however, it can be useful to hide the true location of files, particularly if you're sending something of value (e-books, for example). It may be essential to only send files to logged in users too. send_file makes it possible. It sends files in 4096 byte chunks, so even large files can be sent without slowing the system down.
Iterating through page elements with RJS - Changing page elements with RJS is easy, but what if you don't know exactly which elements you want to change, and would instead prefer to address them with CSS queries? You can with RJS's select method. For example: page.select('#items li').each { |item| item.hide } . Powerful stuff!
Check for existence - When doing a Model.find(id), an exception can be returned if the item with an id of 'id' doesn't exist. If you want to avoid this, use Model.exists?(id) first to get a true or false for whether that item exists or not.
Number helpers for common number tasks - All of these number helpers aren't commonly used but provide great shortcuts: number_to_currency(1234567.948) # => $1,234,567.95 or human_size(1234567890) # => 1.1GB or number_with_delimiter(999999999) # => 999,999,999. There are others.
Testing different route configurations easily - with_routing is a test helper that allows you to temporarily override the default 'routes' in routes.rb for test purposes. Demonstration:
with_routing do |set|
  set.draw { set.connect ':controller/:id/:action' }
  assert_equal(
     ['/content/10/show', {}],
     set.generate(:controller => 'content', :id => 10, :action => 'show')
  )
end
You can learn a little more here.
Get lots of info about requests - Checking request.post? and request.xhr? are popular ways to look for POST and AJAX requests, but some of the other request methods are lesser used. For example: request.subdomains can return an array of subdomains that you could use as part of your authentication scheme, request.request_uri returns the full local request URL, request.host returns the full hostname, request.method returns the HTTP method as a lowercase symbol, and request.ssl? returns true if it's an HTTPS / SSL request.
Improving session performance even more than with ActiveRecord - By default, Rails stores sessions on the local file system. Many users change this to using ActiveRecordStore to store sessions in the database. An even faster alternative is to use Memcached to store sessions, but that takes a lot to set up (and isn't available unless you run your own servers, etc). But you can get faster than ActiveRecordStore by using Stefan Kaes' SQLSessionStore. It circumvents the inefficiencies of ActiveRecordStore using his own direct SQL technique to store sessions.
Caching unchanging data at application startup - If you have data that doesn't change between application restarts, cache it in a constant somewhere. For example, you might have a YAML or XML file in /config that stores application configuration data, and you could load it into a constant in environment.rb, making lookups quick and easy application-wide.
Check your views are rendering valid HTML / XHTML - It's not for everyone, but if your output validates as correct HTML / XHTML, it's a sign your views are going to render properly. Scott Raymond has developed a assert_valid_markup test helper that you can use from your functional tests.
Cleaner HTML output testing - Combine why's Hpricot HTML parser and a special test extension, and you can have powerful tests like so: assert_equal "My title", tag('title') or assert element('body').should_contain('something'). This might be ideal for developing tests to test user built templates. In any case, it's nicer than assert_tag!
Run long-running tasks separately in the background - BackgrounDRb is a small framework, by Ezra Zygmuntowicz, that runs as a daemon in the background that can accept tasks your Rails application sends to it, and whose execution is totally separate to your Rails app. It's extremely powerful, and useful for many tasks such as sending hundreds of e-mails, fetching URLs, and other things you don't want to slow down the request times for your main app. One great demo is to develop a task that increments a variable by 1 and sleeps for 1 second. You can then make a Rails method that queries the variable, and see the distinct separation. Learn more.
Make ids in URLs more user friendly - Override the to_param method on your model and return something like "#{id}-#{title.gsub(/[^a-z0-9]+/i, '-')}" to get URLs like so: http://yoursite.com/posts/show/123-post-title-goes-here .. Much nicer for users, and you don't need to change anything with Post.find(params[:id]) as the non numeric characters will be stripped automagically! Get a full explanation here.
Separate out slices of functionality into Engines - Everyone's heard of Rails' plugins, but pitifully few are using Rails Engines! Rails Engines are like plugins on steroids. They can contain their own models, controllers, and views, and integrate with any applications you run them under. This allows you to split out common fragments of functionality (login, user management, content management, etc.) into separate 'engines' to use in your different projects within minutes. No more writing dull login code! Rails Engines is a big deal, but it should be a far bigger deal.
Calculations - Do you want to get maximums, minimums, averages, or sums for data in your tables? ActiveRecord's Calculations make these all possible. Person.average('age'), Person.maximum(:age, :group => 'last_name'), and Order.sum('total') all become a reality. Most can be customized pretty deeply with extra options, so go read about them if they're not already part of your code.
XML or YAML output of your data - It's not necessarily to create a Builder .rxml template for all XML output. ActiveRecord has a to_xml method that will output the object or result set in XML format. It works with simple objects, to complete tables (like User.find(:all).to_xml). Using includes works too, as with Post.find(:all, :include => [:comments]).to_xml. YAML is also supported, by using to_yaml instead.

Another credit-card-as-Authentication hack

This is a pretty impressive social engineering story: an attacker compromised someone's GoDaddy domain registration in order to change his e-mail address and steal his Twitter handle. It's a complicated attack.
My claim was refused because I am not the "current registrant." GoDaddy asked the attacker if it was ok to change account information, while they didn't bother asking me if it was ok when the attacker did it. [...]
It's hard to decide what's more shocking, the fact that PayPal gave the attacker the last four digits of my credit card number over the phone, or that GoDaddy accepted it as verification.
The misuse of credit card numbers as authentication is also how Matt Honan got hacked.

HOWLERMONKEY: NSA Exploit of the Day

Today's item from the NSA's Tailored Access Operations (TAO) group implant catalog:
HOWLERMONKEY (TS//SI//REL) HOWLERMONKEY is a custom Short to Medium range impant RF Tranceiver. It is used in conjumction with a digital core to provide a complete implant.
(TS//SI//REL) HOWLERMONKEY is a COTS-based transceiver deigned to be compatible with CONJECTURE/SPECULATION networks and STRIKEZONE devices running a HOWLERMONKEY personality. PCB layouts are tailored to individual implant space requirements and can vary greatly in form factor.
Status: Available -- Delivery 3 months
Unit Cost: 40 units: $750/ each, 25 units: $1,000/ each
Page, with graphics, is here. General information about TAO and the catalog is here.

Side-channel attacks on frog calls

The male tĂșngara frog Physalaemus pustulosus uses calls to attract females. But croaking also causes ripples in the water, which are eavesdropped on -- both by rival male frogs and frog-eating bats.

Catalog of snowden revelations

This looks to be very good.
Add that to these three indexes of NSA source material, and these two summaries.
This excellent parody website has a good collection of all the leaks, too.

Thursday, January 30, 2014

Trying to value online privacy

Interesting paper: "The value of Online Privacy," by Scott Savage and Donald M. Waldman.
Abstract: We estimate the value of online privacy with a differentiated products model of the demand for Smartphone apps. We study the apps market because it is typically necessary for the consumer to relinquish some personal information through "privacy permissions" to obtain the app and its benefits. Results show that the representative consumer is willing to make a one-time payment for each app of $2.28 to conceal their browser history, $4.05 to conceal their list of contacts, $1.19 to conceal their location, $1.75 to conceal their phone's identification number, and $3.58 to conceal the contents of their text messages. The consumer is also willing to pay $2.12 to eliminate advertising. Valuations for concealing contact lists and text messages for "more experienced" consumers are also larger than those for "less experienced" consumers. Given the typical app in the marketplace has advertising, requires the consumer to reveal their location and their phone's identification number, the benefit from consuming this app must be at least $5.06.

GINSU: NSA exploit of the day

Today's item from the NSA's Tailored Access Operations (TAO) group implant catalog:
GINSU (TS//SI//REL) GINSU provides software application persistence for the CNE implant, KONGUR, on target systems with the PCI bus hardware implant, BULLDOZER.
(TS//SI//REL) This technique supports any desktop PC system that contains at least one PCI connector (for BULLDOZER installation) and Microsoft Windows 9x, 2000, 20003, XP, or Vista.
(TS//SI//REL) Through interdiction, BULLDOZER is installed in the target system as a PCI bus hardware implant. After fielding, if KONGUR is removed from the system as a result of an operation system upgrade or reinstall, GINSU can be set to trigger on the next reboot of the system to restore the software implant.
Unit Cost: $0
Status: Released / Deployed. Ready for Immediate Delivery
Page, with graphics, is here. General information about TAO and the catalog is here.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

Wednesday, January 29, 2014

First Android bootkit malware found on over 350,000 mobile devices:

A new trojan for Android has been discovered that resides in the memory of infected devices and launches itself early on in the OS loading stage. Even if some elements of the threat are removed successfully, at least one component will still reside in the protected memory area and will reinstall the malware after a reboot and, thus, re-infect the system.
The threat, detected as “Android.Oldboot.1” by Russian security firm Doctor Web, is believed to be the first bootkit for Android according to the company. A bootkit, for those who don’t know, is a kernel-mode rootkit variant that can infect startup code in order to attack even attack full disk encryption systems as well as minimize the possibility that it will be deleted without tampering with the device’s file system.
The malware has reportedly been detected on more than 350,000 mobile devices around the world, including Spain, Italy, Germany, Russia, Brazil, the US and some Southeast Asian countries. 92 percent of the compromised devices are, however, located in China:



This is key to understanding that this threat doesn’t spread by simply browsing the Web, opening attachments, or even sideloading suspicious apps. The malware either ships with the device in question or is manually deployed by someone with physical access to an Android phone or tablet.
Doctor Web explains a device is first infected by placing one of trojan’s components into the boot partition of the file system and by modifying the init script responsible for the initialisation of OS components. When the device is turned on, the modified script loads the code responsible for the trojan’s Linux-library imei_chk, which extracts the files libgooglekernel.so and GoogleKernel.apk, placing them in /system/lib and /system/app, respectively.

Thus, part of the Trojan Android.Oldboot is installed as a typical application which further functions as a system service and uses the libgooglekernel.so library to connect to a remote server and receive various commands, most notably, to download, install or remove certain applications. Reflashing a device with modified firmware that contains the routines required for the trojan’s operation is the most likely way this threat is introduced, according to Doctor Web.
The good news is that your device is unlikely to be infected with Android.Oldboot.1 unless you have purchased a dubious product from a Chinese manufacturer. On the flipside, this sets a dangerous precedent for Android malware that is very difficult to remove.

Full list of infected applications published by “Myournet”:

  • Falling Down
  • Super Guitar Solo
  • Super History Eraser
  • Photo Editor
  • Super Ringtone Maker
  • Super Sex Positions
  • Hot Sexy Videos
  • Chess
  • äž‹ć æ»šçƒ_Falldown
  • Hilton Sex Sound
  • Screaming Sexy Japanese Girls
  • Falling Ball Dodge
  • Scientific Calculator
  • Dice Roller
  • èșČ避ćŒč球
  • Advanced Currency Converter
  • App Uninstaller
  • ć‡ äœ•æˆ˜æœș_PewPew
  • Funny Paint
  • Spider Man
  • 蜘蛛䟠
Full list of infected applications published by “Kingmall2010″:
  • Bowling Time
  • Advanced Barcode Scanner
  • Supre Bluetooth Transfer
  • Task Killer Pro
  • Music Box
  • Sexy Girls: Japanese
  • Sexy Legs
  • Advanced File Manager
  • Magic Strobe Light
  • è‡Žć‘œç»è‰ČçŸŽè…ż
  • ćąšæ°ŽćŠć…‹Panzer Panic
  • èŁžć„”ć…ˆç”ŸMr. Runner
  • èœŻä»¶ćŒșćŠ›ćžèœœ
  • Advanced App to SD
  • Super Stopwatch & Timer
  • Advanced Compass Leveler
  • Best password safe
  • 掷éȘ°ć­
  • ć€šćœ©ç»˜ç”»

Full list of infected apps under the developer name “we20090202″:

  • Finger Race
  • Piano
  • Bubble Shoot
  • Advanced Sound Manager
  • Magic Hypnotic Spiral
  • Funny Face
  • Color Blindness Test
  • Tie a Tie
  • Quick Notes
  • Basketball Shot Now
  • Quick Delete Contacts
  • Omok Five in a Row
  • Super Sexy Ringtones
  • ć€§ćź¶æ„æ‰ŸèŒŹ
  • æĄŒäžŠæ›ČæŁçƒ
  • æŠ•çŻźé«˜æ‰‹

The politics of fear

This is very good:
...one might suppose that modern democratic states, with the lessons of history at hand, would seek to minimize fear ­ or at least minimize its effect on deliberative decision-making in both foreign and domestic policy. But today the opposite is frequently true. Even democracies founded in the principles of liberty and the common good often take the path of more authoritarian states. They don’t work to minimize fear, but use it to exert control over the populace and serve the government’s principle aim: consolidating power.
[...]
However, since 9/11 leaders of both political parties in the United States have sought to consolidate power by leaning not just on the danger of a terrorist attack, but on the fact that the possible perpetrators are frightening individuals who are not like us. As President George W. Bush put it before a joint session of Congress in 2001: "They hate our freedoms: our freedom of religion, our freedom of speech, our freedom to vote and assemble and disagree with each other." Last year President Obama brought the enemy closer to home, arguing in a speech at the National Defense University that "we face a real threat from radicalized individuals here in the United States" -- radicalized individuals who were "deranged or alienated individuals ­- often U.S. citizens or legal residents."
The Bush fear-peddling is usually considered the more extreme, but is it? The Obama formulation puts the “radicalized individuals” in our midst. They could be American citizens or legal residents. And the subtext is that if we want to catch them we need to start looking within. The other is among us. The pretext for the surveillance state is thus established.


Warning: Fake  FileZilla FTP client with login stealer

 

Malware code can be very small, and the impact can be very severe! The Antivirus firm AVAST spotted a malicious version of the open source FTP (File Transfer Protocol) software 'FileZilla' out in the wild.

The software is open source, but has been modified by the hackers that steal users' credentials, offered on various hacked sites for download with banner or text ads.
Once installed, the software's appearance and functionalities are equal to the original version, so a user cannot distinguish between the fake or real one, and the malware version of the “.exe” file is just slightly smaller than the real one.
"The installed malware FTP client looks like the official version and it is fully functional! You can’t find any suspicious behavior, entries in the system registry, communication or changes in application GUI."
The only difference is that the malware version use 2.46.3-Unicode and the official installer use v2.45-Unicode, as shown:
FileZilla Malware Password Stealer
"We found a hardcoded connection detail stealer after deeper analysis. Malware authors abuse open source code and add their own stealer function to the main code."

The modified version copies the login information of the user and sends it to a server that is apparently in Germany, and same IP address of the server hosts three other domains, which are also associated with malware and spam activities.
"Login details are sent to attackers from the ongoing FTP connection only once. Malware doesn't search bookmarks or send any other files or saved connections," Avast explains.
Malicious version of FTP Software FileZilla stealing users' Credentials
This malicious version has been compiled way back in September 2012, and is still detected by just a couple of Antivirus solutions. In the past, Cyber Criminals also used Google Adsense to promote malicious software or the modified open source softwares.
Be Careful when downloading the FileZilla FTP client, such malware could also be employed for spreading more malware. Users are recommended to downloaded the softwares from the official website only.

TAWDRYYARD: NSA Exploit of the Day

Back in December, Der Spiegel published a lot of information about the NSA's Tailored Access Operations (TAO) group, including a 2008 catalog of hardware and software "implants." Because there were so many items in the catalog, the individual items didn't get a lot of discussion. By highlighting an individual implant every day, my goal is to fix that.
Today's item:
TAWDRYYARD (TS//SI//REL TO USA,FVEY) Beacon RF retro-reflector. Provides return when illuminated with radar to provide rough positional location.
(U) Capabilities
(TS//SI//REL TO USA,FVEY) TAWDRYYARD is used as a beacon, typically to assist in locating and identifying deployed RAGEMASTER units. Current design allos it to be detected and located quite easily within a 50' radius of the radar system being used to illuminate it. TAWDRYYARD draws as 8 mu;A at 2.5V (20mu;W) allowing a standard lithium coin cell to power it for months or years. The simplicity of the dsign allows the form factor to be tailored for specific operational requirements. Future capabilities being considered are return of GPS coordinates and a unique target identifier and automatic processing to scan a target area for presence of TWDRYYARDs. All components are COTS and so are non-attributable to NSA.
Concept of Operation
(TS//SI//REL TO USA,FVEY) The board generates a square wave operating at a preset frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal, the illuminating signal is amplitude-modulated (AM) with the square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the clock signal. Typically, the fundamental is used to indicate the unit's presence, and is simply displayed on a low frequency spectrum analyzer. TAWDRYYARD is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
Unit Cost: $30
Status: End processing still in development.
Page, with graphics, is here. General information about TAO and the catalog is here.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

US Privacy and Civil Liberties Oversight Board (PCLOB) Condemns NSA Mass Surveillance

Now we know why the president gave his speech on NSA surveillance last week; he wanted to get ahead of the Privacy and Civil Liberties Oversight Board.
Last week, it issued a report saying that NSA mass surveillance of Americans is illegal and should end. Both EPIC and EFF have written about this.
What frustrates me about all of this -- this report, the president's speech, and so many other things -- is that they focus on the bulk collection of cell phone call records. There's so much more bulk collection going on -- phone calls, e-mails, address books, buddy lists, text messages, cell phone location data, financial documents, calendars, etc. -- and we really need legislation and court opinions on it all. But because cell phone call records were the first disclosure, they're what gets the attention.

EU might raise fines for data breaches

This makes a lot of sense.
Viviane Reding dismissed recent fines for Google as "pocket money" and said the firm would have had to pay $1bn under her plans for privacy failings. Ms Reding said such punishments were necessary to ensure firms took the use of personal data seriously.
And she questioned how Google was able to take so long to getting round to changing its policy.
"Is it surprising to anyone that two whole years after the case emerged, it is still unclear whether Google will amend its privacy policy or not?" she said in a speech.
Ms Reding, who is also vice-president of the European Commission, wants far tougher laws that would introduce fines of up to 5% of the global annual turnover of a company for data breaches.
If fines are intended to change corporate behavior, they need to be large enough so that avoiding them is a smarter business strategy than simply paying them.

Tuesday, January 28, 2014

SPARROW II: NSA exploit of the day

Today's item from the NSA's Tailored Access Operations (TAO) group implant catalog:
SPARROW II (TS//SI//REL) An embedded computer system running BLINDDATE tools. Sparrow II is a fully functional WLAN collection system with integrated Mini PCI slots for added functionality such as GPS and multiple Wireless Network Interface Cards.
(U//FOUO) System Specs
Processor: IBM Power PC 405GPR
Memory: 64MB (SDRAM), 16MB (FLASH)
Expansion: Mini PCI (Up to 4 devices) supports USB, Compact Flash, and 802.11 B/G
OS: Linux (2.4 Kernel)
Application SW: BLINDDATE
Battery Time: At least two hours
(TS//SI//REL) The Sparrow II is a capable option for deployment where small size, minimal weight and reduced power consumption are required. PCI devices can be connected to the Sparrow II to provide additional functionality, such as wireless command and control or a second or third 802.11 card. The Sparrow is shipped with Linux and runs the BLINDDATE software suite.
Unit Cost: $6K
Status: (S//SI//REL) Operational Restrictions exist for equipment deployment.
Page, with graphics, is here. General information about TAO and the catalog is here.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

Monday, January 27, 2014

New Security Risks for Windows XP Systems

Microsoft is trying to stop supporting Windows XP. The problem is that a majority of ATMs still use that OS. And once Microsoft stops issuing security updates to XP, those machines will become increasingly vulnerable.
Although I have to ask the question: how many of those ATMs have been keeping up with their patches so far?
We have far to go with our security of embedded systems.

Saturday, January 25, 2014

PHOTOANGLO: NSA Exploit of the Day

Today's item from the NSA's Tailored Access Operations (TAO) group implant catalog:
PHOTOANGLO (TS//SI//REL TO USA,FVEY) PHOTOANGLO is a joint NSA/GCHQ project to develop a new radar system to take the place of the CTX4000.
(U) Capabilities
(TS//SI//REL TO USA,FVEY) The planned capabilities for this system are:

  • Frequency range: 1 - 2 GHz, which will be later extended to 1 - 4 GHz
  • Maximum bandwidth: 450 MHz.
  • Size: Small enough to fit into a slim briefcase.
  • Weight: Less than 10 lbs.
  • Maximum Output Power: 2W
  • Output:
  • Video
  • Transmit antenna
  • Inputs:
  • External oscillator
  • Receive antenna
(U) Concept of Operation
(TS//SI//REL TO USA,FVEY) TS//SI//REL TO USA,FVEY) The radar unit generates an un-modulated, continuous wave (CW) signal. The oscillator is either generated internally, or externally through a signal generator or cavity oscillator. The unit amplifies the signal and sends it out to an RF connector, where it is directed to some form of transmission antenna (horn, parabolic dish, LPA, spiral). The signal illuminates the target system and is re-radiated. The receive antenna picks up the re-radiated signal and directs the signal to the receive input. The signal is amplified, filtered, and mixed with the transmit antenna. The result is a homodyne receiver in which the RF signal is mixed directly to baseband. The baseband video signal is ported to an external BNC connector. This connects to a processing system, such as NIGHTWATCH, an LFS-2, or VIEWPLATE, to process the signal and provide the intelligence.
Unit Cost: $40k (planned)
Status: Development. Planned IOC is 1st QTR FY09.
Page, with graphics, is here. General information about TAO and the catalog is here.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

Friday Squid Blogging: Giant Squid Caught by Japanese Fisherman

It's big: 13 feet long.
The fisherman was stunned to discover the giant squid trapped in his net, having been caught at a depth of around 70m, about two-thirds of a mile from the coast.
As usual, you can also use this squid post to talk about the security stories in the news that I haven't covered.

Friday, January 24, 2014

Income Inequality as a Security Issue

This is an interesting way to characterizing income inequality as a security issue:
…growing inequality menaces vigorous societies. It is a proxy for how effectively an elite has constructed institutions that extract value from the rest of society. Professor Sam Bowles, also part of the INET network, goes further. He argues that inequality pulls production away from value creation to protecting and securing the wealthy's assets: one in five of the British workforce, for example, works as "guard labour" -- in security, policing, law, surveillance and forms of IT that control and monitor. The higher inequality, the greater the proportion of a workforce deployed as guard workers, who generate little value and lower overall productivity."

Introduction to Vagrant on Linux

Modern operating systems, including Linux, have developed over recent years to bring operating system virtualization to the humble PC. Although virtualization itself isn’t new technology and Linux isn’t the only OS that can run virtual machines, the combination of open source technologies along with mature hypervisors means that Linux is able to offer advanced virtualization solutions. Among these solutions is Vagrant – a tool which works with virtualization software like Virtual Box and VMware to provide an easy way to create, configure and reproduce virtual machines with a known state. The ability to easily create multiple virtual machines in a known configuration can help with testing, software development and deployment. It is also useful in small offices or at home (for hobbyists) as test machines can be easily created and allowing experimentation with new software or configurations.
What Vagrant does is take a base guest operating system, say Ubuntu Linux, and create virtual machines that can be started, stopped and destroyed with simple commands. When  a virtual machine is initially created, it can be put into a predefined state – software can be installed, files can be created or copied onto it and other administration tasks can be performed.
To install Vagrant on Linux, visit the downloads page and pick the correct download for your distribution. The project offers packages for Debian and Ubuntu as well as Red Hat, CentOS and Fedora. There are 32-bit and 64-bit versions available. If you aren’t sure whether you have installed a 32-bit or 64-bit version of Linux then use “uname -a” to find out.
For Ubuntu you can install the .deb file using:

sudo dpkg -i vagrant_1.4.3_x86_64.deb
 
Replace “vagrant_1.4.3_x86_64.deb” with the version that you downloaded.
For Vagrant to work, you need to install VirtualBox (or VMware). Download the correct version for your distribution from the VirtualBox Downloads page. Oracle provides packages for Ubuntu, Debian, SUSE, Fedora and others.
For Ubuntu you can install the VirtualBox .deb file using:

sudo dpkg -i virtualbox-4.3_4.3.6-91406~Ubuntu~precise_amd64.deb
 
Replace “virtualbox-4.3_4.3.6-91406~Ubuntu~precise_amd64.deb” with the version that you downloaded.
The quickest way to get Vagrant up and running is to create a directory for the virtual machine (VM) and then run two Vagrant commands, one to initialize the system and one to start the VM. First create a directory and cd into it:

mkdir vagrant-test1
cd vagrant-test1
 
Now initialize Vagrant:

vagrant init precise32 http://files.vagrantup.com/precise32.box
 
This tells Vagrant to initialize and use “precise32″ as the base operating system (called a box). The third parameter tells Vagrant where it can find the files for that box. In case you aren’t familiar with Ubuntu’s code names, precise32 means Ubuntu 12.04 LTS 32-bit. Vagrant also offers official boxes for Ubuntu 12.04 LTS 64-bit (precise64) and Ubuntu 10.04 LTS 32-bit and 64-bit (lucid32 and lucid64). There is also a list of community provided boxes at  http://vagrantbox.es.
As part of the initialization phase, Vagrant will create a file called “Vagrantfile” in the current working directory (i.e. vagrant-test1). You can create as many directories and initialize Vagrant as many times as you like. Each directory and Vagrantfile represent one virtual machine. These different VMs can be based on the same box (i.e. precise32) or on different boxes.
To start the VM run:

vagrant up
 
vagrant up - first time
If this is the first time that a virtual machine has been run with the selected box, then Vagrant will download the .box file from the URL provided and boot it. This may take a few minutes, depending on the speed of your Internet connection, but subsequent boots will be much quicker as the .box file is only downloaded once. If you get a warning about the “guest additions” versions not matching, you can safely ignore it. Any files in the VMs working directory on the host machine will be available in the “/vagrant” directory on the VM. By default, that means just the Vagrantfile.
Once booted, the virtual machine is running in a headless mode (without a monitor or virtual screen) and so you need to connect to it via SSH. You can do this via a normal SSH client from any other machine on your network, or you can use the built-in ssh command. If you want to use another SSH client, note that the correct IP address is the IP address of the host machine (the PC running Vagrant and VirtualBox) but on a different port. During the boot up Vagrant will show how port 22 (for SSH) has been forwarded, it will likely be to port 2222.
To use the built-in ssh command, type:

vagrant ssh
 
You are now connected to the VM. To leave the SSH connection, type “exit” or press “CTRL + D”.
vagrant ssh
To stop a running VM, use

vagrant halt
 
and to delete the VM, use

vagrant destroy
 
When a VM is destroyed, the base operating system (from the .box file) remains stored internally in Vagrant and further VMs can be started whenever necessary without Vagrant downloading the .box file again.
In just two commands (“vagrant init precise32...” and “vagrant up“), Vagrant allowed you to boot up a fully functional, SSH accessible virtual Linux machine. Advanced configuration happens via the Vagrantfile and you can find more details in the Vagrant documentation.

NIGHTWATCH: NSA Exploit of the Day

Today's item from the NSA's Tailored Access Operations (TAO) group implant catalog:
NIGHTWATCH (TS//SI//REL TO USA,FVEY) NIGHTWATCH is a portable computer with specialized, internal hardware designed to process progressive-scan (non-interlaced VAGRANT signals).
(U) Capability Summary
(TS//SI//REL TO USA,FVEY) The current implementation of NIGHTWATCH consists of a general-purpose PC inside of a shielded case. The PC has PCI digitizing and clock cards to provide the needed interface and accurate clocking required for video reconstruction. It also has:
  • horizontal sync, vertical sync and video outputs to drive an external, multi-sync monitor.
  • video output
  • spectral analysis up to 150 kHz to provide for indications of horizontal and vertical sync frequencies.
  • frame capture and forwarding
  • PCMCIA cards for program and data storage
  • horizontal sync locking to keep the display set on the NIGHTWATCH display.
  • frame averaging up to 2^16 (65536) frames.
(U) Concept of Operation
(TS//SI//REL TO USA,FVEY) The video output from an appropriate collection system, such as a CTX4000, PHOTOANGLO, or general-purpose receiver, is connected to the video output on the NIGHTWATCH system. The user, using the appropriate tools either within NIGHTWATCH or externally, determines the horizontal and vertical sync frequencies of the targeted monitor. Once the user matches the proper frequencies, he activates "Sync Lock" and frame averaging to reduce noise and improve readability of the targeted monitor. If warranted, the user then forwards the displayed frames over a network to NSAW, where analysts can look at them for intelligence purposes.
Unit Cost: N/A
Status: This system has reached the end of its service life. All work concerning the NIGHTWATCH system is strictly for maintenance purposes. This system is slated to be replaced by the VIEWPLATE system.
Page, with graphics, is here. General information about TAO and the catalog is here.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

Consumer Manipulation

Tim Harford talks about consumer manipulation:
Consider, first, confusion by design: Las Vegas casinos are mazes, carefully crafted to draw players to the slot machines and to keep them there. Casino designers warn against the "yellow brick road" effect of having a clear route through the casino. (One side effect: it takes paramedics a long time to find gamblers in cardiac arrest; as Ms SchĂŒll also documents, it can be tough to get the slot-machine players to assist, or even to make room for, the medical team.) Most mazes in our economy are metaphorical: the confusion of multi-part tariffs for mobile phones, cable television or electricity. My phone company regularly contacts me to assure me that I am on the cheapest possible plan given my patterns of usage. No doubt this claim can be justified on some narrow technicality but it seems calculated to deceive. Every time I have put it to the test it has proved false.
I recently cancelled a contract with a different provider after some gizmo broke. The company first told me the whole thing was my problem, then at the last moment offered me hundreds of pounds to stay. When your phone company starts using the playbook of an emotionally abusive spouse, this is not a market in good working order.
This is a security story: manipulation vs. manipulation defense. One of my worries about our modern market system is that the manipulators have gotten too good. We need better security -- either technical defenses or legal prohibitions -- against this manipulation.

Thursday, January 23, 2014

NIGHTSTAND: NSA Exploit of the Day

Today's device from the NSA's Tailored Access Operations (TAO) group implant catalog:
NIGHTSTAND (TS//SI//REL) An active 802.11 wireless exploitation and injection tool for payload /exploit delivery into otherwise denied target space. NIGHTSTAND is typically used in operations where wired access to the target is not possible.
(TS//SI//REL) NIGHTSTAND - Close Access Operations • Battlefield Tested • Windows Exploitation • Standalone System
System Details
  • (U//FOUO) Standalone tool currently running on an x86 laptop loaded with Linux Fedora Core 3.
  • (TS//SI//REL) Exploitable Targets include Win2k, WinXP, WinXPSP1, WINXPSP2 running Internet Explorer versions 5.0-6.0.
  • (TS//SI//REL) NS packet injection can target one client or multiple targets on a wireless network.
  • (TS//SI//REL) Attack is undetectable by the user.
(TS//SI//REL) Use of external amplifiers and antennas in both experimental and operational scenarios have resulted in successful NIGHTSTAND attacks from as far away as eight miles under ideal environmental conditions.
Unit Cost: Varies from platform to platform
Status: Product has been deployed in the field. Upgrades to the system continue to be developed.
Page, with graphics, is here. General information about TAO and the catalog is here.
Presumably, the NSA can use this "injection tool" in all the same ways it uses QUANTUM. For example, it can redirect users to FOXACID servers in order to attack their computers.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

Questioning the Efficacy of NSA's Bulk-Collection Programs

Two reports have recently been published questioning the efficacy of the NSA's bulk-collection programs. The first one is from the left-leaning New American Foundation (report here, and one-page tabular summary here).
However, our review of the government’s claims about the role that NSA “bulk” surveillance of phone and email communications records has had in keeping the United States safe from terrorism shows that these claims are overblown and even misleading. An in-depth analysis of 225 individuals recruited by al-Qaeda or a like-minded group or inspired by al-Qaeda’s ideology, and charged in the United States with an act of terrorism since 9/11, demonstrates that traditional investigative methods, such as the use of informants, tips from local communities, and targeted intelligence operations, provided the initial impetus for investigations in the majority of cases, while the contribution of NSA's bulk surveillance programs to these cases was minimal. Indeed, the controversial bulk collection of American telephone metadata, which includes the telephone numbers that originate and receive calls, as well as the time and date of those calls but not their content, under Section 215 of the USA PATRIOT Act, appears to have played an identifiable role in initiating, at most, 1.8 percent of these cases. NSA programs involving the surveillance of non-U.S. persons outside of the United States under Section 702 of the FISA Amendments Act played a role in 4.4 percent of the terrorism cases we examined, and NSA surveillance under an unidentified authority played a role in 1.3 percent of the cases we examined.
The second is from Marshall Erwin of the right-leaning Hoover Institute (report here, and summary here).
My conclusion is simple: neither of these cases demonstrates that bulk phone records collection is effective. Those records did not make a significant contribution to success against the 2009 plot because at the point at which the NSA searched the bulk records database, the FBI already had sufficient information to disrupt the plot. It is also unlikely that bulk collection would have helped disrupt the 9/11 attacks, given critical barriers to information sharing and as demonstrated by the wealth of information already available to the intelligence community about al-Mihdhar.

Wednesday, January 22, 2014

List of minimalist frameworks

A list of minimalist frameworks (simple and lightweight). All in alphabetical order.

Index

Framework for CSS

Use http://refresh-sf.com/yui/ to compress, and get size "after compression and gzip"/1024 = KiB
Name Minimum size (base) Repository License
Atatonic 5.1 KiB (minified) http://code.google.com/p/atatonic-css-framwork/ MIT
Cascade Framework 10.6 KiB (all modules) https://github.com/jslegers/cascadeframework MIT
Cascade Framework Light 2.0 KiB (all modules) https://github.com/jslegers/cascadeframeworklight MIT
Concrete 3.2 KiB https://github.com/davidlumley/concrete MIT
KNACSS 13.2 KiB (all modules) https://github.com/raphaelgoetter/KNACSS WTFPL
Kube 15.2 KiB https://github.com/sashka/kube Unknown
Min 0.9 KiB (minified and gzipped) https://github.com/owenversteeg/min MIT
Mistype 2.1 KiB https://github.com/ZDroid/mistype MIT
PocketGrid 0.5 KiB https://github.com/arnaudleray/pocketgrid MIT
Pure 4.3 KiB (gzipped) https://github.com/yui/pure BSD
RocketCSS 6.4 KiB https://github.com/RocketCSS/RocketCSS MIT
Simple Grid 1.1 KiB https://github.com/ThisIsDallas/Simple-Grid Unknown
Toast 1.2 KiB https://github.com/daneden/Toast Unknown

Web Framework for C

Name Repository License
Raphters https://github.com/DanielWaterworth/Raphters GPL

Frameworks for Front-end JS

Name Bower Install Repository License
Aura bower install aura https://github.com/aurajs/aura MIT
Kraken Unknown https://github.com/cferdinandi/kraken MIT
Min.js Unknown https://github.com/remy/min.js MIT
Reactive.coffee bower install reactive-coffee https://github.com/yang/reactive-coffee MIT
Responsive Unknown https://github.com/JimBobSquarePants/Responsive MIT
ScaleApp bower install scaleapp https://github.com/flosse/scaleApp MIT
Spine bower install spine https://github.com/spine/spine MIT
Stapes bower install stapes http://github.com/hay/stapes MIT

Web framework for Go

Name Repository License
Beego Framework https://github.com/astaxie/beego Apache License V2
Gocraft/web https://github.com/gocraft/web MIT
Martini https://github.com/codegangsta/martini MIT
Traffic https://github.com/pilu/traffic MIT

Web framework for Haskell

Name Cabal Install Repository License
Scotty cabal install scotty https://github.com/scotty-web/scotty BSD3
Snap cabal install snap https://github.com/snapframework/snap BSD3

Web framework for Java

Name Repository License
Play Framework https://github.com/playframework/playframework Apache License V2
Restlet https://github.com/restlet/restlet-framework-java Apache License V2
RestX https://github.com/restx/restx Apache License V2
Spark https://github.com/perwendel/spark Apache License V2
Stapler https://github.com/stapler/stapler BSD

Web framework for Javascript

Name Repository License
Meteor https://github.com/meteor/meteor MIT

Web framework for Node.js

Name NPM Install Repository License
Express.js [sudo] npm install express https://github.com/visionmedia/express MIT
Codekart npm install codekart https://github.com/myworld4059/Codekart MIT
Flatiron [sudo] npm install -g flatiron https://github.com/flatiron MIT
Hapi npm install hapi https://github.com/spumko/hapi Unknown
Koa [sudo] npm install koajs https://github.com/koajs/koa MIT
Meryl npm install meryl https://github.com/kadirpekel/meryl MIT
Partial.js [sudo] npm install -g partial.js https://github.com/petersirka/partial.js MIT
Restify npm install restify https://github.com/mcavage/node-restify MIT
SocketStream [sudo] npm install -g socketstream https://github.com/socketstream/socketstream MIT
Sails.js [sudo]npm -g install sails https://github.com/balderdashy/sails MIT

Web framework for Perl

Name cpan install Repository License
Dancer cpan Dancer https://github.com/PerlDancer/Dancer Artistic or GPL v.1
Mojolicious cpan Mojolicious https://github.com/kraih/mojo Artistic 2.0

Web framework for PHP

Name Repository License
Afro https://github.com/jbrooksuk/Afro MIT
AuraPHP https://github.com/auraphp/Aura.Web_Kernel BSD
Bullet https://github.com/vlucas/bulletphp MIT
Deano http://github.com/colindean/deano MIT
Fat Free https://github.com/bcosca/fatfree GPL 3
Flight https://github.com/mikecao/flight MIT
Hackwork https://github.com/ZDroid/hackwork MIT
Limonade https://github.com/sofadesign/limonade Unknown
MicroMVC https://github.com/Xeoncross/MicroMVC MIT
Nanite https://github.com/nirix/nanite LGPLv3
Phalcon Framework https://github.com/phalcon/cphalcon BSD3
polyframework https://github.com/polymedio/polyframework BSD3
Respect\Rest http://github.com/Respect/Rest BSD3
Silex https://github.com/silexphp/Silex MIT
Slim https://github.com/codeguy/Slim MIT
Swiftlet https://github.com/ElbertF/Swiftlet MIT
Yaf https://github.com/laruence/php-yaf PHP License v3.01

Web framework for Python

Name PIP Install Repository License
Bobo pip install bobo https://github.com/zopefoundation/bobo ZPL 2.1
Bottle [sudo] pip install bottle https://github.com/defnull/bottle MIT
CherryPy pip install CherryPy https://bitbucket.org/cherrypy/cherrypy/overview BSD
Cyclone pip install cyclone https://github.com/fiorix/cyclone Apache License 2.0
Flask pip install Flask https://github.com/mitsuhiko/flask BSD
Itty-Bitty pip install itty https://github.com/toastdriven/itty/ BSD3
Klein pip install klein https://github.com/twisted/klein MIT
ObjectWeb Unknown https://github.com/aisola/ObjectWeb LGPLv3
Pecan pip install pecan https://github.com/stackforge/pecan BSD
Pyramid pip install pyramid https://github.com/Pylons/pyramid BSD-derived
Tornado pip install tornado https://github.com/facebook/tornado Apache License 2.0
Web.py [sudo] pip install web.py https://github.com/webpy/webpy Public domain
Wheezy.web pip install wheezy.web https://bitbucket.org/akorn/wheezy.web MIT

Web framework for Ruby

Name Gem Install Repository License
Brooklyn gem install brooklyn https://github.com/luislavena/brooklyn MIT
Camping gem install camping http://github.com/camping/camping MIT
Cuba gem install cuba https://github.com/soveran/cuba MIT
Grape gem install grape https://github.com/intridea/grape MIT
Hobbit gem install hobbit https://github.com/patriciomacadden/hobbit MIT
Kenji gem install kenji https://github.com/kballenegger/Kenji Azure
Nancy gem install nancy https://github.com/guilleiguaran/nancy MIT
New York, New York gem install nyny https://github.com/alisnic/nyny MIT
Padrino gem install padrino https://github.com/padrino/padrino-framework MIT
Ramaze gem install ramaze https://github.com/ramaze/ramaze MIT
Sinatra gem install sinatra https://github.com/sinatra/sinatra/ MIT

Web framework for Scala

Name Repository License
Finatra https://github.com/twitter/finatra Apache License V2
Play Framework https://github.com/playframework/playframework Apache License V2
Scalatra https://github.com/scalatra/scalatra BSD
Spray https://github.com/spray/spray Apache License V2

Web framework for .NET (C#)

Name Repository License
Aurora https://github.com/frankhale/aurora GPL
Nancy https://github.com/NancyFx/Nancy/ MIT

LOUDAUTO: NSA Exploit of the Day

Today's item from the NSA's Tailored Access Operations (TAO) group implant catalog:
LOUDAUTO (TS//SI//REL TO USA,FVEY) Audio-based RF retro-reflector. Provides room audio from targeted space using radar and basic post-processing.
(U) Capabilities
(TS//SI//REL TO USA,FVEY) LOUDAUTO's current design maximizes the gain of the microphone. This makes it extremely useful for picking up room audio. It can pick up speech at a standard, ofice volume from over 20' away. (NOTE: Concealments may reduce this distance.) It uses very little power (~15 uA at 3.0 VDC), so little, in fact, that battery self-discharge is more of an issue for serviceable lifetime than the power draw from this unit. The simplicity of the design allows the form factor to be tailored for specific operation requirements. All components at COTS and so are non-attributable to NSA.
(U) Concept of Operation
(TS//SI//REL TO USA,FVEY) Room audio is picked up by the microphone and converted into an analog electrical signal. This signal is used to pulse position modulate (PPM) a square wave signal running at a pre-set frequency. This square wave is used to turn a FET (field effect transistor) on and off. When the unit is illuminated with a CW signal from a nearby radar unit, the illuminating signal is amplitude-modulated with the PPM square wave. This signal is re-radiated, where it is picked up by the radar, then processed to recover the room audio. Processing is currently performed by COTS equipment with FM demodulation capability (Rohde & Schwarz FSH-series portable spectrum analyzers, etc.) LOUDAUTO is part of the ANGRYNEIGHBOR family of radar retro-reflectors.
Unit Cost: $30
Status: End processing still in development
Page, with graphics, is here. General information about TAO and the catalog is here.
This one is kind of cool, I think.
In the comments, feel free to discuss how the exploit works, how we might detect it, how it has probably been improved since the catalog entry in 2008, and so on.

Tuesday, January 21, 2014

Adware Vendors Buy and Abuse Chrome Extensions

This is not a good development:
To make matters worse, ownership of a Chrome extension can be transferred to another party, and users are never informed when an ownership change happens. Malware and adware vendors have caught wind of this and have started showing up at the doors of extension authors, looking to buy their extensions. Once the deal is done and the ownership of the extension is transferred, the new owners can issue an ad-filled update over Chrome's update service, which sends the adware out to every user of that extension. [...]
When malicious apps don't follow Google's disclosure policy, diagnosing something like this is extremely difficult. When Tweet This Page started spewing ads and malware into my browser, the only initial sign was that ads on the Internet had suddenly become much more intrusive, and many auto-played sound. The extension only started injecting ads a few days after it was installed in an attempt to make it more difficult to detect. After a while, Google search became useless, because every link would redirect to some other webpage. My initial thought was to take an inventory of every program I had installed recently -- I never suspected an update would bring in malware. I ran a ton of malware/virus scanners, and they all found nothing. I was only clued into the fact that Chrome was the culprit because the same thing started happening on my Chromebook -- if I didn't notice that, the next step would have probably been a full wipe of my computer.