Telescope Array Composite Satellite Map using Landsat Data
GIS, GDAL, rasterio, LandSat 8, AWS, NumPy, Telescope Array, Remote Sensing, Python,
The Telescope Array Cosmic Ray Observatory (TA) is the largest cosmic ray detector in the northern hemisphere. To better illustrate the observatory’s location and size for presentations and papers, I created my own composite satellite map. The python GDAL module was used to import the rasters of the different bands, rasterio module was used to set the window on the raster, and NumPy (Numerical Python) was used to create a composite color image using the RGB bands. The satellite image was centered on the central laser facility (CLF) at the observatory. The satellite image below was created in this Jupyter python notebook on my Github profile.
- Python and GIS
- Retrieving LandSat 8 Data off AWS
- Creating a composite image from the RGB layers
- Setting the Raster Window
- Haversine Formula and Great-Circle Distances to CLF Coordinates
- TA Satellite Map
- Resources
Python and GIS
Python can be used for Geographic Infromation Systems (GIS) using modules such as GDAL and rasterio. The Geospatial Data Abstraction Library (GDAL) is a library for working with the main forms of geospatial data (or images): rasters or vectors. The python GDAL module allows for importing raster and vector geospatial data in python. rasterio is also a library for interfacing with rasters of geospatial data.
To import GDAL and rasterio in python (and other modules I use in my example):
# Import Python Libraries #
import pandas as pd
import os, glob, wget
import matplotlib.pyplot as plt
from osgeo import gdal
import rasterio
import numpy as np
import math
import re
The python module versions I used in my example are:
Python 3.6.9
Pandas: 0.24.1
Numpy: 1.16.0
Matplotlib: 3.0.2
re: 2.2.1
wget: 3.2
GDAL: 2.4.2
Rasterio: 1.1.5
Retrieving LandSat 8 Data off AWS
The United States Geological Survey (USGS) LandSat 8 satellite continuously images the earth with 11 spectral bands. Information on retrieving the LandSat 8 data (images) from Amazon Web Services (AWS) from this link. First you need the scene list. With the scene list you have access to all the scenes that can be retrieved form AWS.
Reading the scene
f = 'landsat_8_aws_scene_list_20200808'
df = pd.read_csv(f)
# Check Scene List #
print(df.shape)
display(df.tail())
We see shape of the list and the most recent scenes
(2039703, 12)
productId entityId acquisitionDate cloudCover processingLevel path row min_lat min_lon max_lat max_lon download_url
2039698 LC08_L1GT_208004_20200806_20200807_01_RT LC82080042020219LGN00 2020-08-06 11:27:39.935012 98.45 L1GT 208 4 76.76868 17.36822 79.20000 29.91599 https://s3-us-west-2.amazonaws.com/landsat-pds...
2039699 LC08_L1TP_176062_20200806_20200807_01_RT LC81760622020219LGN00 2020-08-06 08:32:59.402007 62.79 L1TP 176 62 -3.94766 23.41179 -1.84473 25.46072 https://s3-us-west-2.amazonaws.com/landsat-pds...
2039700 LC08_L1TP_176079_20200806_20200807_01_RT LC81760792020219LGN00 2020-08-06 08:39:46.324864 0.01 L1TP 176 79 -28.48941 17.69588 -26.37063 20.03246 https://s3-us-west-2.amazonaws.com/landsat-pds...
2039701 LC08_L1TP_192036_20200806_20200807_01_RT LC81920362020219LGN00 2020-08-06 10:01:30.892612 0.00 L1TP 192 36 33.53324 7.12533 35.65588 9.66675 https://s3-us-west-2.amazonaws.com/landsat-pds...
2039702 LC08_L1TP_192044_20200806_20200807_01_RT LC81920442020219LGN00 2020-08-06 10:04:42.059052 29.92 L1TP 192 44 22.04505 4.32692
The Central Laser Facility (CLF) at the Telescope Array Cosmic Ray Observatory is roughly centered between all three fluorescence detector sites. It is used the origin of coordinate system to determine the distance between TA sites relative to the CLF. We will use the CLF as the center of the satellite image we will create. The CLF is located 39.296944, -112.908611. Using the minimum and maximum of each scene, TA was determined to be in scenes with path = 38, row = 33. To select scenes over TA, find low cloud cover, and a spring month for aesthetics (for green fields in Delta, UT).
# Location of the TA Observatory in the Landsat 8 page #
CLF_path = 38
CLF_row = 33
clf_df = df[df['path'] == CLF_path]
clf_df = clf_df[clf_df['row'] == CLF_row]
## Turn acquisition time into datetime ##
clf_df['acquisitionDate'] = pd.to_datetime(clf_df['acquisitionDate'])
## Cut on cloud cover to less than 5% ##
clf_df = clf_df[clf_df['cloudCover'] < 5 ]
## Find a Spring Month (For aesthetics) ##
clf_df = clf_df[clf_df['acquisitionDate'].dt.month == 4]
## Sort by Most Recent Pass ##
clf_df = clf_df.sort_values('acquisitionDate',ascending=False)
display(clf_df.head())
which yields the at list of scenes with the selections above
productId entityId acquisitionDate cloudCover processingLevel path row min_lat min_lon max_lat max_lon download_url
1948587 LC08_L1TP_038033_20200425_20200426_01_RT LC80380332020116LGN00 2020-04-25 18:08:06.916571 3.24 L1TP 38 33 37.80464 -113.83262 39.94594 -111.09092 https://s3-us-west-2.amazonaws.com/landsat-pds...
1971570 LC08_L1TP_038033_20200425_20200509_01_T1 LC80380332020116LGN00 2020-04-25 18:08:06.916571 3.24 L1TP 38 33 37.80464 -113.83262 39.94594 -111.09092 https://s3-us-west-2.amazonaws.com/landsat-pds...
947415 LC08_L1TP_038033_20150428_20170301_01_T1 LC80380332015118LGN01 2015-04-28 18:07:52.232930 2.87 L1TP 38 33 37.80416 -113.85367 39.94543 -111.11137 https://s3-us-west-2.amazonaws.com/landsat-pds...
947414 LC08_L1TP_038033_20150412_20170228_01_T1 LC80380332015102LGN02 2015-04-12 18:07:58.860876 2.49 L1TP 38 33 37.80456 -113.83613 39.94586 -111.09092 https://s3-us-west-2.amazonaws.com/landsat-pds...
Selecting the latest scene that matches our criteria and formatting the download URL
## Download Directory ##
dir = '/GDF/TAResearch/TAmap/Landsat_imgs'
# Select Scene to download #
map_df = clf_df.head(1)
display(map_df)
# Scene Download URL
download_url = str(map_df['download_url'].values[0])
# print download_url
productId = str(map_df['productId'].values[0])
# Format list of Images to download #
download_urls = download_url.replace('/index.html','/{0}'.format(productId))
download_urls = [ '{0}_{1}.TIF'.format(download_urls, band) for band in bands]
# print download_urls[0]
Then the .tif
rasters for all 11 spectral bands is retrieved from AWS using wget()
# Make Image Directory to put pulled images #
img_dir = dir + '/' + productId
if not os.path.exists(img_dir):
os.makedirs(img_dir)
# Retrieve Images from AWS #
image_files = []
print('Saving to {0}'.format(img_dir))
for d in download_urls:
image_file = img_dir + '/' + d.split('/')[-1]
image_files.append(image_file)
# print(image_file)
if not os.path.isfile(img_dir+ '/'+d.split('/')[-1]):
print('Retrieving {0}...'.format(d.split('/')[-1]))
wget.download(d, out=img_dir)
elif os.path.isfile(d.split('/')[-1]): print('{0} already exists...'.format(d.split('/')[-1]))
which retrieves the files for LC08_L1TP_038033_20200425_20200426_01_RT.
LandSat 8 Spectral Bands
Operational Land Imager (OLI)
Band | Type | Wavelength, λ [µm] | Resolution [m] |
---|---|---|---|
1 | Visible | 0.43 - 0.45 | 30 |
2 | Visible | 0.450 - 0.51 | 30 |
3 | Visible | 0.53 - 0.59) | 30 |
4 | Red | 0.64 - 0.67 | 30 |
5 | Near-Infrared | 0.85 - 0.88 | 30 |
6 | SWIR 1 | 1.57 - 1.65 | 30 |
7 | SWIR 2 | 2.11 - 2.29 | 30 |
8 | Panchromatic | 0.50 - 0.68 | 15 |
9 | Cirrus | 1.36 - 1.38 | 30 |
Thermal Infrared Sensor (TIRS)
Band | Type | Wavelength, λ [µm] | Resolution [m] |
---|---|---|---|
10 | TIR 1 | 10.6 - 11.19 | 100 |
11 | TIR 2 | 11.5 - 12.51 | 100 |
Creating a composite image from the RGB layers
To create a composite image using the RGB bands (bands 2-4), the bands are loaded using gdal.Open()
, normalized and values converted to floats, and then stacked in an array using Numpy dstack()
def norm(band):
'''
Normalize Spectrum Bands for Composite Image
'''
band_min, band_max = band.min(), band.max()
return ((band - band_min)/(band_max - band_min))
# Open Bands Using GDAL #
b = gdal.Open(image_files[1])
g = gdal.Open(image_files[2])
r = gdal.Open(image_files[3])
# call the norm function on each band as array converted to float
b2 = norm(b.ReadAsArray().astype(np.float))
b3 = norm(g.ReadAsArray().astype(np.float))
b4 = norm(r.ReadAsArray().astype(np.float))
# Create RGB Composite #
rgb = np.dstack((b4,b3,b2))
# Show Composite Image #
plt.figure(figsize=(16,16))
plt.imshow(rgb)
plt.savefig(productId+'.svg')
plt.savefig(productId+'.png', dpi=300)
Which produces the satellite image
Composite satellite image of LandSat 8 scene LC08_L1TP_038033_20200425_20200426_01_RT
Setting the Raster Window
To center the raster on a point of interest, in this case, the CLF, we use define the following function. We determine the downloaded raster size with raster.RasterXSize
and raster.RasterYSize
, then determine where the point of interest is relative to the corners, the number of pixels, and the pixel size (given the band’s resolution). Then the window is set using rasterio.windows.Window()
and the location of the point of interest and the desired window size.
def centerPOI(max_lat,
min_lat,
max_lon,
min_lon,
raster,
poi_lat = ta_df[ta_df['Site'] == 'CLF']['Latitude'].values[0],
poi_lon = ta_df[ta_df['Site'] == 'CLF']['Longitude'].values[0],
pixels = 2000):
'''
Create a Window for the Rasters to Center on a Point of Interest (POI) within the Raster with given latitude and longitude of site.
poi_lat = POI latitude in Degrees Decimal
poi_lon = POI longitude in Degrees Decimal
p = width of window in pixels. Each Pixel is 30m in width
'''
# Raster Longitude and Latitude Width #
lon_range = max_lat - min_lat
lat_range = max_lon - min_lon
# print(max_lat, min_lat, lon_range)
# print(max_lon, min_lon, lat_range)
# Relative Position of POI #
poi_x = (poi_lon - clf_df['min_lon'].values[0]) / lon_range
poi_y = 1 - (poi_lat - clf_df['min_lat'].values[0]) / lat_range # Flipped y axis #
# Open Raster and Find Size #
raster = gdal.Open(raster)
dx, dy = raster.RasterXSize, raster.RasterYSize
# Find Position of POI in Raster #
poi_x = poi_x * dx
poi_y = poi_y * dy
# Calculate Window for Focus on POI #
# Window Corner #
x_o, y_o = poi_x - .5 * pixels, poi_y - .5 * pixels
# Window Width #
d_x, d_y = pixels, pixels
window = rasterio.windows.Window(x_o, y_o, d_x, d_y)
return window
Using the centerPOI()
to set a square window ± 30km around the CLF
# Create Window #
window = centerPOI(poi_lat = ta_df[ta_df['Site'] == 'CLF']['Latitude'].values[0],
poi_lon = ta_df[ta_df['Site'] == 'CLF']['Longitude'].values[0],
max_lat = map_df['max_lon'].values[0],
min_lat = map_df['min_lon'].values[0],
max_lon = map_df['max_lat'].values[0],
min_lon = map_df['min_lat'].values[0],
raster = image_files[3],
pixels = 2000)
# print(window)
# Load RGB Bands #
r = rasterio.open(image_files[3]).read(1, window=window)
g = rasterio.open(image_files[2]).read(1, window=window)
b = rasterio.open(image_files[1]).read(1, window=window)
pan = rasterio.open(image_files[8]).read(1, window=window)
# Normalize RGB Bands #
b2 = norm(b.astype(np.float))
b3 = norm(g.astype(np.float))
b4 = norm(r.astype(np.float))
b8 = norm(pan.astype(np.float))
print(np.shape(b4))
# Create RGB Composite by stacking layers #
rgb = np.dstack((b4,b3,b2))
print(np.shape(rgb))
yields the figure
Composite satellite image of the Telescope Array Cosmic Ray Observatory. The Town of Delta and the surrounding fields and smaller town is to the right. The northern portion of the Sevier lakebed is to the bottom left.
Zooming in further on the FD sites and the CLF with square windows ± 3 km.
## Create An Image of Each Site Zoomed in ##
fig = plt.figure(figsize=(16,16))
for (i, site, lat, lon) in zip(range(1,len(ta_df)+1),ta_df['Site'].values,ta_df['Latitude'].values,ta_df['Longitude'].values):
plt.subplot(2,2,i)
print(i, site, lat, lon)
## Finding Offset from recorded CLF lat and Long values to Center CLF ##
lat_offset, lon_offset = -.015, -.017
# Create Window #
window = centerPOI(poi_lat = lat + lat_offset,
poi_lon = lon + lon_offset,
max_lat = map_df['max_lon'].values[0],
min_lat = map_df['min_lon'].values[0],
max_lon = map_df['max_lat'].values[0],
min_lon = map_df['min_lat'].values[0],
raster = image_files[3],
pixels = 200)
# Load Bands #
r = rasterio.open(image_files[3]).read(1, window=window)
g = rasterio.open(image_files[2]).read(1, window=window)
b = rasterio.open(image_files[1]).read(1, window=window)
# Normalize Bands #
b2 = norm(b.astype(np.float))
b3 = norm(g.astype(np.float))
b4 = norm(r.astype(np.float))
# Stack RGB for Composite #
rgb = np.dstack((b4,b3,b2))
# Plot Site #
plt.imshow(rgb)
plt.title('{0} Site ({1:.6f}, {2:.6f})'.format(site,lat,lon))
fig.suptitle('TA Sites with LandSat 8 Composite Images (30m Resolution)', fontsize=16)
plt.savefig('ta_sites.svg')
plt.savefig('ta_sites.png', dpi=300)
Composite satellite images of the Telescope Array Cosmic Ray Observatory FD sites and the CLF. Upper Left: CLF. Upper Right: Black Rock. Lower Left: Long Ridge. Lower Right: Middle Drum. Middle Drum is most apparent as it is one of the biggest sites.
Haversine Formula and Great-Circle Distances to CLF Coordinates
To determine the position of the SD sites relative to the center of the satellite for adding these positions to the map, I take the measured GPS locations of the SD and use the Haversine_formula. I use the Haversine formula to determine the positions of the SDs on the map given the components of the great-circle distance between the SD site and the CLF. Then I have the SDs in CLF coordinates and account of the curvature of the Earth. I defined the haversine()
function to compute the great-circle distance and bearing between two positions given their latitude and longitude. I use this in the CLF_coord()
to determine the x and y position in CLF coordinates.
def haversine(longitude_1, latitude_1, longitude_2, latitude_2):
"""
Calculate the great circle distance between two points p1 and p2
on the earth (specified in decimal degrees)
"""
# convert decimal degrees to radians #
longitude_1, latitude_1, longitude_2, latitude_2 = map(math.radians, [longitude_1, latitude_1, longitude_2, latitude_2])
# haversine formula #
dlongitude = longitude_2 - longitude_1
dlatitude = latitude_2 - latitude_1
a = math.sin(dlatitude/2)**2 + math.cos(latitude_1) * math.cos(latitude_2) * math.sin(dlongitude/2)**2
c = 2 * math.asin(math.sqrt(a))
# Earth Radius in m from wikipedia #
radius = [6378100, 6356800, 6378137, 6356752.3141, 6399593.6259,
6371008.7714, 6371007.1810, 6371000.7900, 6378137.0,
6356752.3142, 6399593.6258, 6371008.7714, 6371007.1809,
6371000.7900, 6378137.0, 6356752.314140, 6335439,
6384400, 6352800, 6371230]
description = [
'nominal \"zero tide\" equatorial',
'nominal \"zero tide\" polar ',
'equatorial radius',
'semiminor axis (b)',
'polar radius of curvature (c)',
'mean radius (R1)',
'radius of sphere of same surface (R2)',
'radius of sphere of same volume (R3)',
'WGS-84 ellipsoid, semi-major axis (a)',
'WGS-84 ellipsoid, semi-minor axis (b)',
'WGS-84 ellipsoid, polar radius of curvature (c)',
'WGS-84 ellipsoid, Mean radius of semi-axes (R1)',
'WGS-84 ellipsoid, Radius of Sphere of Equal Area (R2)',
'WGS-84 ellipsoid, Radius of Sphere of Equal Volume (R3)',
'GRS 80 semi-major axis (a)',
'GRS 80 semi-minor axis (b)',
'meridional radius of curvature at the equator',
'Maximum (the summit of Chimborazo)',
'Minimum (the floor of the Arctic Ocean)',
'Average distance from center to surface'
]
R_Earth = {'r': radius, 'description': description}
R_Earth = pd.DataFrame(R_Earth)
# Radius of the Earth in km #
r = (R_Earth['r'][3] + ta_df['Altitude'][0])/ 1000
#r = R_Earth['r'][3]/ 1000
# Bearing Angle CW from North
bearing = math.atan2(math.sin(longitude_2-longitude_1)*math.cos(latitude_2), math.cos(latitude_1)*math.sin(latitude_2)-math.sin(latitude_1)*math.cos(latitude_2)*math.cos(longitude_2-longitude_1))
# Great Circle Distance #
d = c * r
# Convert Bearing Angle to degrees #
# bearing = bearing * 180 / pi
return d, bearing
def CLF_coord(longitude, latitude):
'''
Calculates the x and y components of distance between a a point with decimal degree longitude, latitude.
CLF Coordinates :
+x : East [m]
+y : North [m]
'''
clf_longitude, clf_latitude = ta_df['Longitude'][0], ta_df['Latitude'][0]
d, bearing = haversine(clf_longitude, clf_latitude, longitude, latitude)
# Find the x and y component of the Distance #
x = d * math.sin(bearing)
y = d * math.cos(bearing)
return x, y
TA Satellite Map
Last step is loading the raster with the desired window and overlaying sites and information after calculating their positions relative to the center.
## Finding Offset from recorded CLF lat and Long values to Center CLF ##
lat_offset, lon_offset = -.019, -.02
# Create Window #
window = centerPOI(poi_lat = ta_df[ta_df['Site'] == 'CLF']['Latitude'].values[0] + lat_offset,
poi_lon = ta_df[ta_df['Site'] == 'CLF']['Longitude'].values[0] + lon_offset,
max_lat = map_df['max_lon'].values[0],
min_lat = map_df['min_lon'].values[0],
max_lon = map_df['max_lat'].values[0],
min_lon = map_df['min_lat'].values[0],
raster = image_files[3],
pixels = 2000)
# Load Bands #
r = rasterio.open(image_files[3]).read(1, window=window)
g = rasterio.open(image_files[2]).read(1, window=window)
b = rasterio.open(image_files[1]).read(1, window=window)
# Normalize Bands #
b2 = norm(b.astype(np.float))
b3 = norm(g.astype(np.float))
b4 = norm(r.astype(np.float))
# Create RGB Composite #
rgb = np.dstack((b4,b3,b2))
# FOV Rings #
FOV_x, FOV_y = [], []
FOV_c = []
for (ang_i, ang_f, xo, yo, color) in zip(ta_df['FOVi'].values,ta_df['FOVf'].values,ta_df['X'].values,ta_df['Y'].values,ta_df['Color'].values):
r = 30
x, y = FOV_arc(r, ang_i, ang_f, xo, yo)
FOV_x.append(x)
FOV_y.append(y)
FOV_c.append(color)
## Composite Array Plot ##
fig = plt.figure(figsize=(16,16))
# Satellite Background #
ax=fig.add_subplot(111)
ax.imshow(rgb)
ax.tick_params(size=0)
ax.set_aspect('equal')
ax.set_yticklabels([])
ax.set_xticklabels([])
# FD Site Back Grounds #
clr_clf = 'red'
clr_fd = '#065398'
clr_sd = 'white'
ax2 = fig.add_subplot(111, frame_on=False)
# Plot FOV #
for (x,y,c) in zip(FOV_x, FOV_y, FOV_c):
if x[0] != 0 :
alpha = .2
ax2.fill(x,y,color=clr_fd,alpha=alpha)
# FD Sites #
# for (lat,lon,color,label) in zip(ta_df['Latitude DD'],ta_df['Longitude DD'],ta_df['Color'],ta_df['Site']):
# x, y = CLF_coord(lon,lat)
for (x,y,label) in zip(ta_df['X'],ta_df['Y'],ta_df['Site']):
if label == 'CLF' : ax2.scatter(x,y,marker='h', s=90, color=clr_clf)
else : ax2.scatter(x,y,marker='^', s=90, color=clr_fd)
# SD Sites #
for (x,y) in zip(sd_df['X'],sd_df['Y']):
ax2.scatter(x,y,marker='.', s=90, color=clr_sd)
# Label Sites #
for (x,y,text) in zip(ta_df['X'],ta_df['Y'],ta_df['Site']):
#print (x,y,color,label)
if text != 'CLF':
if text == 'MD':
ax2.text(x,y + 1.25,text, verticalalignment='center', horizontalalignment='center', fontsize=16, color='white')
else :
ax2.text(x,y - 1.25,text, verticalalignment='center', horizontalalignment='center', fontsize=16, color='white')
ax2.set_xlim([-30,30])
ax2.set_ylim([-30,30])
ax2.tick_params(size=0)
ax2.set_yticklabels([])
ax2.set_xticklabels([])
# Stuff for Legend #
ax2.scatter(40,40,marker='^',color=clr_fd, label='FD Site')
ax2.scatter(40,40,marker='h',color=clr_clf, label='CLF Site')
ax2.scatter(40,40,marker='.',color=clr_sd, label='SD Site')
# Scale #
ax2.plot([-20,-10],[27,27],color='white')
ax2.text(-15, 28, '10 km', verticalalignment='center', horizontalalignment='center', color='white', fontsize=14)
# North Compass #
ax2.plot([-25,-25],[22,27],color='white')
ax2.text(-25, 28, 'North', verticalalignment='center', horizontalalignment='center', color='white', fontsize=14)
ax2.fill([-25,-25.5,-24.5],[27,26,26],color='white')
## Attribution ##
ax.text(0.98, 0.02, 'Greg Furlich\n USGS Landsat 8\n Telescope Array Cosmic Ray Observatory Satellite Map',
verticalalignment='bottom', horizontalalignment='right',
transform=ax.transAxes,
weight='medium',
color='white', fontsize=15)
# Plot Info #
# plt.title('Telescope Array Cosmic Ray Observatory')
# plt.xlabel('East [km]')
# plt.ylabel('North [km]')
leg =plt.legend(loc='upper right', bbox_to_anchor=(0.98, 0.98), prop={'size':16}, markerscale=2, framealpha = .2)
frame = leg.get_frame()
frame.set_color('white')
frame.set_linewidth(0)
for text in leg.get_texts():
plt.setp(text, color = 'w')
ax2.set_aspect('equal')
# plt.show()
plt.savefig('ta_map.svg', bbox_inches='tight', pad_inches=-.1)
plt.savefig('ta_map.png', bbox_inches='tight', pad_inches=-.1, dpi=300)
The final result is the composite satellite map to show the Telescope Array Cosmic Ray Observatory.
Satellite map of TA using Landsat 8 data.
Resources
RGB Stacked Satellite Images in Python
Back to Top