Archive for the ‘amazon’ tag
In Retrospect: Lessons & Tips from a Large Federal Implementation
Last year, I presented on “Esri WebGIS Platform – How we implemented ArcGIS, and you can too” at FedGIS. This year, I shared another summary – lessons and tips from that implementation. That is especially helpful if you are dealing with the unique security responsibilities of the federal government around high-value PII/PHI-based data assets and Expedited Life Cycle (XLC) processes.
From a technical perspective, I shared how we implemented a hybrid and disconnected ArcGIS design inside a 3-zone architecture with multi-VPN and multi-NIC networks on Red Hat Enterprise Linux.
From a high-level management perspective, I shared how that played out inside the federal environment.
Esri WebGIS Platform
Customer Need, Deployment Option, Authority to Operate (ATO), Challenges, Solutions, Lessons
Esri in AWS Cloud
Geodata Based Decisions
How to Use Location Analytics to Improve the Effectiveness of Public-Facing Sites
HowTo: Run ‘ArcGIS for Server Advanced Enterprise’ (10.3.1) on Amazon EC2 Red Hat Enterprise Linux (7)
The talks on ArcGIS Server at ESRI Health GIS were fun, but I wanted more – specifically, to install and administer its latest release on Amazon Web Services, all via the trusted command line. Here’s how I did that:
To follow along, get an EDN license and an AWS account. Especially, if you have been in the industry for long, there’s no good excuse to not have those with the biggest companies in GIS and da Cloud (and while you are at it, get MapBox and CartoDB accounts too).
$ ssh -i "key.pem" ec2-user@#.#.#.#.compute.amazonaws.com
$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 7.1 (Maipo)
$ sudo yum upgrade
$ sudo yum update
$ sudo yum install emacs
$ sudo emacs ~/.bashrc
force_color_prompt=yes
$ df -h
> ...
$ lsblk
> NAME SIZE TYPE MOUNTPOINT
> xvda 20G disk
> |_xvda2 6G part /
$ sudo yum install gdisk
$ sudo gdisk /dev/xvda/
$ print
$ delete
$ new
$ ####
$ \r
$ #
$ print
$ write
$ y
$ sudo xfs_growfs /
$ sudo file -s /dev/xvda
$ sudo reboot
$ tar -xvf ArcGIS_for_server_linux_1031_145870.gz
$ ./Setup
$ sudo emacs /etc/security/limits.conf
$ ec2-user soft nofile 65535
$ ec2-user hard nofile 65535
$ ec2-user soft nproc 25059
$ ec2-user hard nproc 25059
$ ulimit -Hn -Hu
$ ulimit -Sn -Su
$ ./Setup
$ locate -i authorization.ecp
$ readlink -f authorization.ecp
$ ./authorizeSoftware -f /path/authorization.ecp
$ ./authorizeSoftware -s
$ ./startserver.sh
$ netstat -lnp | grep "6080"
$ hostname
$ emacs /etc/hosts
$ 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
$ ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
$ #.#.#.# localhost localhost.localdomain localhost4 localhost4.localdomain4
Custom TCP rule TCP 6080 0.0.0.0/0
//#.#.#.# or machinename:6080/arcgis/manager
ArcGIS Server Setup Wizard -> Create New Site
Primary Site Administrator -> Create Account
Root Server Directory: /home/username/arcgis/server/usr/directories
Configuration Store: /home/username/arcgis/server/usr/config-store
$ cat /home/ec2-user/arcgis/server/usr/logs/EC2/server/server-...log
> ...
>
>
>
...
$ sudo yum list installed
$ sudo yum install wget
$ wget http://vault.centos.org/6.2/os/x86_64/Packages/xorg-x11-server-Xvfb-1.10.4-6.el6.x86_64.rpm
$ sudo yum localinstall xorg-x11-server-Xvfb-1.10.4-6.el6.x86_64.rpm
sudo yum install Xvfb
sudo yum install freetype
sudo yum install fontconfig
sudo yum install mesa-libGL
sudo yum install mesa-libGLU
sudo yum install redhat-lsb
sudo yum install glibc
sudo yum install libXtst
sudo yum install libXext
sudo yum install libX11
sudo yum install libXi
sudo yum install libXdmcp
sudo yum install libXrender
sudo yum install libXau
$ sudo yum clean all
$ cd /tmp/
$ sudo rm -r *
$ logout
//#.#.#.#:6080/arcgis/manager/
//#.#.#.#:6080/arcgis/admin/
Custom TCP rule TCP 6443 0.0.0.0/0
$ ./stopserver.sh
$ ./uninstall_ArcGISServer
Conclusion: 6443 or 8443?
After years of doing this with first ESRI (PROD), then MapServer (PROD) and GeoServer (DEV), I went back to the dark ahem ESRI side. And what do I keep finding? That the big two are blending together in terms of looks. E.g. The console of the other Java-powered mapping server, GeoServer, is looking similar to that of its big brother on-steroids. The third, MapServer, somewhat paradoxically on the other hand, has both come a long way (MapCache and ScribeUI, yay!) and still lost ground.
Next up, testing Tippecanoe.
PS:
* I tried both 10.3.1 and 10.0 on Ubuntu (15.04), unsupported. While both installed, site creation didn’t work because of missing packages – searching through apt-cache didn’t help either. On Windows, there is always their CloudBuilder.
Related:
* GeoNet
* Landsat on AWS in ArcGIS
#HealthGIS: Notable links and final thoughts on the conference
Health websites using ESRI ++
* With ArcGIS JavaScript
• CDC’s Division for Heart Disease and Stroke Prevention (DHDSP) Atlas
* With ArcGIS Server / ArcGIS Online (via Apache Flex)
• HealthLandscape’s Accountable Care Organization (ACO) Explorer
• Dartmouth’s Atlas (try generate KML)
• NMQF’s Methicillin-resistant Staphylococcus aureus (MRSA) mapping
• HRSA’s Datawarehouse
Health websites whose global participants have trouble with software licenses ++
* With OpenLayers and DHIS2 (~ an opensource InstantAtlas)
• PEPFAR’s (a president’s best legacy) Data for Accountability, Transparency and Impact (DATIM) – coming soon to GeoServer + MapLoom and OpenLayers
* Even Highmaps(!)
• NCHS’s Health Indicators Warehouse (HIW)
* Many More…
Dots
Clearly, there’s no shortage of health data or technologies, esp. following ACA’s requirements of uniform data collection standards, just a continuing kerfuffle with overlaying disparate JSON/OGC tiles from their many data owners and manifold service endpoints. Unfortunately, only part of this problem is technical. Take Flu mapping, for instance. CDC, WHO, WebMD (with MapBox) and Google, even Walgreens does it. Or take HIV mapping where you can choose from CDC and NMQF, among others. Even anonymized private claims data is available for a couple of Ks a month. I think a bigger part of the problem is the misalignment between vendors’ business interests and mandates of various agencies and goals of the health research community at large.
Connect
At some point, researchers and epidemiologists would want to see how these data tiles correlate to each other. And GIS professionals would want a quicker way to ‘overlay this layer’ with out having to dig through Firebug. And compress it over the wire, while you are at it (when our users in remote Africa were asked to switch off their smartphones to view desktop maps, we understood data compression a little differently).
Crunch
And then they would want to analyze them, be it on the server with Big Data or in the client with smaller ones. On analyses, your favorite GIS continues to take heat from tools like Tableau among conference attendees.
Overall, a growing use of ArcGIS Server’s publisher functionalities and a compelling body of story map templates leveraging its narrative text capabilities. E.g. Atlas for Geographic Variation within Medicare. On publishing, I suspect some researchers would like to see a Mapbox plugin for QGIS. Yes, you can render and uploads maps from TileMill to your Mapbox account, but CartoDB has QgisCartoDB where you can view, create, edit or delete data from QGIS to your CartoDB account (I needn’t add that Python-powered QGIS remains a favorite among matplotlib-loving researchers).
PS: My ranking of how easy it is to connect to federal health datasets –
1. CDC (E.g. NCHS, Wonder, Health Indicators)
2. CMS (E.g. DNAV, Medicare – try Hospital Compare – Info, Spreadsheet, JSON)
3. HRSA (E.g. Datawarehouse).
Related:
* CDC’s GIS Resources
* CDC’s Submit Maps
* Hospital Referral Region (HRR) – A regional market area for tertiary medical care
* Health Savings Account (HSA) – A tax-advantaged medical savings account available to some taxpayers
++ While log analyses attest that mono-themed web maps provide a better user experience, given the nature of health data and the costs behind spinning off another mapp (yup, blended words to make a portmanteau), sometimes you just have to combine themes.
Mobile Browsers
As the Google-backed Open Handset Alliance takes shape, I have been testing dominant WAP browsers on my 2-year old touchscreen PocketPC. This resulting post should narrow down the choices for those who follow:
Deep Fish by Microsoft appears to be the most promising of the lot. Unfortunately, it is in a strict testing phase and no longer accepting registrations. Until then, you can always make do with Internet Explorer for Mobile.
Opera, arguably the slimmest desktop browser out there, has a paid version- Opera Mobile for $24. But if you do not have a smartphone and/or do not wish to spend any money, try Opera Mini.
The Mozilla Foundation has the amusingly named Minimo.
Opera Mobile offers tab-browsing like Minimo, and does a better job at handling pop-ups and javascripts than Internet Explorer. And like Minimo, it offers ‘grab and drag’ navigation thus eliminating scrollbars. Opera Mobile also offers subtle other improvements, like allowing you to change your User Agent- a must-have for those websites that recognize mobile browsers, but remain inexplicably unprepared for them. On the other hand, Minimo features XUL [try this in Firefox – chrome://browser/content/browser.xul] that has impressively found its way into Mozilla Amazon Browser etc, and is the most customizable.
Absent from all these is the Nokia Web Browser– the sometime favorite of opensource mobile development. After all, its early emulators are what helped a lot of programmers/developers gain a handle on mobile development long before Google.
–π
Related:
Follow Up [1]: Wireless Application Protocol
Wanted: Proactive Policies
>> WAP
News:: Spatial
News:: Science & Technology
Sample *.xul
xda-developers Forum
Picsel Browser
Zumobi
Proxy Server
Mini-Me
Follow Up [4]: Graphic Software
Yet more evidence of acceptance of Google Maps and through it, of spatial relevance, by established publications:
A Guide to Commuting and Readers’ Stories
How Much Is Gas In Jersey?
In a related development, Microsoft continues to play catch-up with Google by acquiring GeoTango. However, with its “3D Internet Visualization- a truly open and web services-oriented solution”, GeoTango may just be the partner Microsoft needs for a tango.
–π
Related:
ESRI ArcWeb Services
NASA World Wind
Follow Up [3]: Graphic Software
This week Yahoo released its own take on online mapping. Its new service includes both Flash and AJAX APIs coupled with the ability to geocode.
If you think about it, sooner or later this had to happen- developers finally mustering the courage to embrace arty Macromedia Flash for distributing spatial information in a big way, like Geocentric. Actually, Google has been using Flash for a different distribution for quite some time now. But this release by Yahoo and its under-1000 dollar price-tag should help Flash emerge as a more visible player in the online mapping game.
Did the earlier musings portend this?
–π
Related:
Yahoo Developer Network
GeoCool! Tutorial
Google Local, MSN Virtual Earth, Amazon A9, AOL MapQuest
Application: Google Earth
Discussion Forum
Follow Up [1]: Graphic Software
It is good to know that some professionals concur with the views expressed in my earlier post on the potential for graphic software, like Macromedia Flash. One comment links to an impressive demonstration of this largely untapped potential.
Anyway, two companies whose product GUI I enjoy interfacing with- Adobe and Macromedia, announced their merger earlier this month.
Both their flagship products have become industry-standards in exchanging documents and creating experience-rich applications across platforms. The largely unused spatial potential within Macromedia Flash combined with the increasingly widespread use of Adobe PDF/SVG maps and the sprouting of some exciting derivatives like geoPDF, pstoedit and GSview, make this merger important to how spatial information is exchanged in the near future.
Graphic Software
The discussion “So …How About That Election Coverage?” at Directions Magazine makes you think about graphic software, like Macromedia Flash, that cater to small-time spatial needs.
Such graphic software, minus the topology and advanced query benefits, function well as basic spatial tools and comfortably serve data over the web with a “fair” amount of interactivity.
Does this make your overpriced IMS overhyped and overblown too?
[my comment]
Macromedia Flash fills this niche quite well as demonstrated [here]. And as the market seems to indicate, it does that [while] satisfying more customers than what an overly fancy GIS would. [This] reminds me of the MapQuest survey when polled customers had expressed great contentment with their level of map detail, whereas cartographers were red with indignation. Akin to using an atomic clock to serve your wake-up call- not needed!
[/my comment]
So is the complexity in Geospatial, better still Spatial, Information System or SIS overblown too? Much of SIS requires common-sense logic arranged linearly. If a person can drive her car in rush-hour traffic as she deciphers vague directions off a schematic map while trying to make sense of rain-washed road signs and maintain a semblance of conversation with her passenger, and still manage to engage the kid in the back-seat [read multi-linear tasking]; she can achieve a sound understanding of spatial databases with little persistence, except for the eye-for-details that comes with practice.
My point: SIS is non-complex and not at the cutting-edge of technological change, and there is ample room for non-traditional spatial software!
PS:
This rise of non-traditional spatial software challenges the accepted definition of SIS. If you were to follow the modernists approach to design where in the end you remove everything you can without taking away from the essence of your creation and apply it to defining a SIS, you wonder what such a conceptual SIS would be in its simplest stark-naked Spartan form?
Loading ...