Redshift Blog Post With a Performance and Cost Lean…
Some of this is a repeat for my own value from another of my blog posts Read More
Redshift and PostgreSQL Schemas Versus Owners
Coming from an Oracle background where schemas are roughly equivalent to owners the important differences between Read More
Redshift create plain user, superuser, createdb user etc…
select * from pg_user_info where usename like ‘mcost%’; create user mcost with createuser password ‘1234Mike’; create Read More
Big Data Blog Posts That I Found Interesting – I am sure there are many others
Best Of Amazon AWS Big Data Blog https://aws.amazon.com/blogs/big-data/orchestrate-multiple-etl-jobs-using-aws-step-functions-and-aws-lambda/ https://github.com/aws-samples/aws-etl-orchestrator https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/best-practices.html#organizingstacks https://aws.amazon.com/blogs/big-data/build-a-data-lake-foundation-with-aws-glue-and-amazon-s3/ https://aws.amazon.com/blogs/big-data/orchestrate-apache-spark-applications-using-aws-step-functions-and-apache-livy/
psql to the rescue – a client program for postgreSQL or Redshift
Here are SOME links to some psql features: http://www.postgresqltutorial.com/psql-commands/ http://www.postgresqltutorial.com/postgresql-list-users/ http://www.postgresqltutorial.com/postgresql-describe-table/ The most current PostgreSQL docs. Read More
Preparing For The Amazon AWS Big Data Specialty Certification Test
Here is some of the material I used to prepare for the Amazon AWS Big Data Read More
installing scala on RHEL 7 or clones and do we really need this with Spark? No!
# you can install this Scala anywhere – but I install it with sudo / root Read More
Hive 2 Connection Woes …
This post is mostly for my reference but, if you are having trouble connecting to a Read More
tcph-kit to generate data
TCPH – http://www.tpc.org/tpch/ Here is where you get the kit to generate data – https://github.com/gregrahn/tpch-kit # Read More
setting up java with java_home jre_home and path
# installing the java jdk via .tar.gz from java.oracle.com to a RHEL / CentOS / Oracle Read More
installing and using hadoop and spark from scratch singlenode and cluster
While this post is a bit backwards (the installation is at the bottom) – it has Read More
Windows 10 Sound / Volume Is Not Consistent
Was driving me crazy – this fixed it. I found an even more simple solution here. The Read More
MongoDB Review / Summary 4.0
MongoDB Review as of Version 4.0 August 2018 MongoDB Overview MongoDB is referred to as a Read More
Using AWS EMR and Spark to Perform ETL
https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-1/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-2-code-development-with-notebooks-and-docker/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-3-running-pyspark-on-emr/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-4-analysing-the-data/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-5/ http://spark.apache.org/docs/latest/index.html
Machine Learning Categorizations and Techniques
Note Ok, I’m obviously new to machine learning – and this post is mostly just for Read More
Redshift data ingestion deduplication / upsert ( delete / insert ) using a staging table – and a little tuning
Why would you do this? In Redshift blocks are immutable – and are re-written completely – Read More
Problem with my new HP x360 trackpad being too sensitive using trackpad taps
I had a major problem with my new HP x360 trackpad being too sensitive using trackpad Read More
References to AWS Docs. and git / github For Redshift tuning table design distribution styles and distribution keys and much more…
References to AWS Docs. For Redshift tuning table design distribution styles and distribution keys and much Read More
redshift diststyle and distkey combination define slice distributions
diststyle and distkey combination define slice distributions diststyle defaults to “even” when not defined distkey has Read More
Find the many version of software – like Java installed with “alternatives” on linux
alternatives –config java + 1 /usr/lib/jvm/jre-1.6.0-openjdk/bin/java * 2 /etc/alternatives/java_sdk/bin/java 3 /usr/lib/jvm/jre-1.5.0-gcj/bin/java 4 /usr/java/latest/bin/java 5 /usr/java/jdk1.7.0/bin/java