Monday, March 21, 2016

GDS Technologies - Energy from Water?

I've been watching GDS Technologies for a while now (http://gdstechnologies.ca/) and I am intrigued with the idea of generating power using their system and water alone.  To me, it sounds too good to be true, although the underlying technology has been investigated for a while now.

Just recently, their website had the banner:


The first 2500 units of the GDS5000 will be released and ready for delivery by July 5, 2016


But as of today (March 21, 2016) they removed it from their website.  Does that mean the technology does not work, or is it just a delay in the manufacturing process?

The other thing that concerns me is their disclaimer:


Our generators are for emergency backup use only. For warranty purpose, maximum run time is 4 continuous hours per day.

It makes me wonder if it's just a glorified battery that runs for 4 hours and then is recharged?

I'm optimistic about energy generation and alternatives, but this company does raise a lot of questions...

Wednesday, October 29, 2014

Netflix Asgard 1.5 Deployments

With the upgrade for v1.5 of Asgard Netflix, the API for deployments has changed and not all of the old endpoints exist (specifically /cluster/deployment for one).  Because of this, we have had to upgrade our deployment plan to use the new APIs.  However, there does not seem to be a lot of documentation out there for the new APIs, so I thought I'd put together some information in hopes it might help others in the future.

The primary API endpoint for deployments in 1.5 is:

http://<host>/<region>/deployment

where <host> is the host, and <region> is the EC2 region, such as us-west-2.  So a full deployment endpoint might look something like this:

http://asgard.jonfenner.com/us-west-2/deployment

Steps for a deployment are:
  1. Prepare for a deployment
  2. Start a deployment

Prepare for a deployment

Endpoint: http://<host>/<region>/deployment/prepare?id=<asg>

This gets the ASG json information which can be used in the deployment process

Start a deployment

Endpoint: http://<host>/<region>/deployment/start

The deployment consists of "steps".  We've implemented the following:

- Create the new ASG (always starts out with 0 instances)
- Resize it to the appropriate # of instances
- Disable the old ASG
- Delete the old ASG

Each step is self-checking, so if it fails, none of the succeeding steps will execute.

Below is a sample python script to implement this:

#!/usr/bin/python

import sys
import urllib2
import jsonbr /> import requests

version = '1.0'

print 'AMI Asgard Deployment Script V' + version

asgardhost = 'localhost:8080'
ec2region = 'us-west-2'
baseurl = 'http://' + asgardhost + '/' + ec2region + '/deployment'
notify = 'xxx@yyy.com'

if (len(sys.argv) != 3):
   print 'Syntax: launch.py <ASG Id> <AMI Id>'
   sys.exit()

asgid=sys.argv[1]
amiid=sys.argv[2]

print 'Asgard Host: ' + asgardhost
print 'EC2 Region: ' + ec2region
print 'ASG: ' + asgid
print 'AMI to Launch: ' + amiid

query = baseurl + '/prepare?id=' + asgid
f = urllib2.urlopen(query)
deflcjson = f.read()
f.close()

deflc = json.loads(deflcjson)

deflc['lcOptions']['imageId'] = amiid

deflc['deploymentOptions'] = {
    "clusterName": asgid,
    "notificationDestination": notify,
    "steps": [
      { "type": "CreateAsg" },
      { "type": "Resize", "targetAsg": "Next", "capacity": deflc['asgOptions']['minSize'], "startUpTimeoutMinutes": 41},
      { "type": "DisableAsg", "targetAsg": "Previous" },
      { "type": "DeleteAsg", "targetAsg": "Previous" },
    ]
  }

posturl = baseurl + '/start'
headers = {'Content-type': 'application/json', 'Accept': 'text/plain'}
response = requests.post(posturl, data=json.dumps(deflc), headers=headers)

print response
print response.text

Also, be sure you've implemented Eureka and healthchecks for all services.  Asgard waits for both a Eureka UP and a positive healthcheck.

Monday, November 11, 2013

Information Analysis #1

Category: Cell phones

Information Analysis is the study of information and what is represents.  You might also know it under the terms of data mining, information gathering, intelligence gathering, or intelligence assessment, just to name a few.

This blog entry focuses on a method for un-anonymizing celluar usage based on an actors tendency to use multiple devices for legitimate vs. illegitimate activities.

Scenario: The primary actor (PA) runs a legitimate business and uses cell phone 1 (CP1) for daily communication of business.  This phone is registered to PA.  This takes place in multiple locations through the city where PA lives.  PA also runs an illegal money laundering business and uses his legitimate business to cover for it.  In order to keep the two separate, he uses a prepaid cell phone (CP2) for all money laundering business.  The assumption is that by using a prepaid cell phone, it will not be able to be linked back to him.

Task: Given that we suspect the PA is conducting illegal activities, we want to be able to tie those activities back to PA2.

Solution: The solution lies in the use of cellular towers and the logging of information related to phone calls.  Assuming that cell phone activity for a given tower or set of towers can be obtained, a cross reference algorithm is devised which will link the activity of the PA between cell towers and phones.

For example, during any particular day, the PA makes calls on CP1 which is covered by cell sites 449, 2132, and 474.  Since we know that CP1 is registered to the PA, we can track these activities:

Date/Time Cell Site Originating Number Destination Number Duration (minutes)
2013-11-10 08:02:33 449 802-310-1234 817-467-3311 5
2013-11-10 08:32:18 2132 802-310-1234 802-846-2111 5
2013-11-10 10:12:12 2132 802-310-1234 202-233-3232 5
2013-11-10 13:11:22 474 802-310-1234 802-355-2314 5
2013-11-10 17:30:01 449 802-310-1234 603-453-1234 5

Now to identify the CP2 usage, we focus on the cell sites that have been used by the PA during this day.  If we plot out the cell site usage, time, and location we can create a probability map of activity for the PA with the probability that they are within range of a certain cell site at any given time.  This is based solely on a linear distribution of probability between any two sites based on the progression of time, and the straight-line connectivity between the cell sites.

Once we have the probability map defined, then we can take a look at all of the other cell phone calls from those cell sites during the time period and prescribe a probability that one of those phones is CP2.

What the algorithm takes into consideration is based on the probability of time that the PA is within access to any one cell site, the phone calls from cell numbers to that site and that time will be assigned a probability.  If we look at the summed probabilities of the cell numbers across all the sites, then we can establish which other cell phones have a high probability of being CP2.

Once CP2 has been identified, a wiretap order can be executed to obtain the information required for prosecution.

Conclusion: This algorithm works best under the following scenarios:
  • The PA travels around enough to use multiple cell sites for both CP1 and CP2
  • The PA uses both cell phones multiple times during the day
  • The PA does not have both cell phones on all the time during the day
Of course, the alternative if both cell phones are on all the time during the day is to just cross-correlate registration of the cells phones with the local towers.  Any two that register at or near the same time have a high probability of being with the PA.

While not a 100% solution, this algorithm provides a high probability of locating multiple linked cell phones for specific scenarios.


Friday, October 4, 2013

Security Tip #1

In hopes of sharing some of my knowledge in the Network and Security realm, I thought it would be fun to post tips.  With that in mind, here is

Security Tip #1

Category: Site Security 

For anyone who runs a website, site security is something that you are going to have to address in one way or another.  This might be proactively looking at the site from an external user point of view, or responding to a security incident in which you site has been hacked or compromised in some fashion.

Like all security, be it network, physical, or personal, there is no one silver bullet that is going to handle all of your needs.  For site security, this means addressing things such as:

  • Physical security of the system on which the website is housed
  • Network security of how the system is connected to the outside world
  • System security for the operating system that is is running on, including making sure security patches and subsystem components are up-to-date.
  • Site compartmentalization, i.e., if someone breaks in to one part of the site, how much access might they easily gain to other parts (database, other servers, etc.).
This first tip focuses on what you can do about system security of your site.  Doing an online scan of your site will help determine if any malware or other malicious code is attached to your site in anyway.  This is a good indicator if your site has been compromised.  Some good online scanners include:
They all have free options for running them manually, which you can easily do on a regular basis.  Or, if you have a larger number of domains or want deeper scans for your websites, most have paid options as well.

Wednesday, October 2, 2013

Getting around the Government Shutdown websites

So, if you're like me you've found quite a number of government websites that are down due to the government shutdown.  This is very annoying if you're looking for information on one of these websites and can't get to it. However, there is a solution for most of the static data... Google Cache.

One of the benefits of Google scouring the web is that they keep cached versions of many of the documents and web pages for a period of time.  To access those, just click on the "cached" version when you do a Google search.  Here's an example of a search I did on Google:



Notice the little caret next to the return URL in the search list?  Just click on that and you get the option to view the cached version.

While it's not ideal (PDFs are converted to HTML, for example), at least it may get you some of the information you're looking for.

Hope that was helpful!

Jon