Analyzing How Your Redshift Cluster is Performing Queries and Storing Data
Analyzing How Your Redshift Cluster is Performing Queries and Storing Data Useful Queries Space Used – Read More
Federated Active Directory Integration With Redshift – other blog posts and documentation reference
https://aws.amazon.com/blogs/big-data/federate-database-user-authentication-easily-with-iam-and-amazon-redshift/
Verizon’s secret $40 unlimited plan is now available to everyone – you just need an iPhone
https://www.yahoo.com/tech/verizon-secret-40-unlimited-plan-now-available-everyone-182849186.html originally posted on bar.com Earlier this year, it emerged that Verizon is very stealthily funding Read More
Redshift static (reboot required) versus dynamic – no reboot – parameters
The following WLM properties are dynamic: Enable short query acceleration Maximum run time for short queries Read More
Redshift Blog Post With a Performance and Cost Lean…
Some of this is a repeat for my own value from another of my blog posts Read More
Redshift and PostgreSQL Schemas Versus Owners
Coming from an Oracle background where schemas are roughly equivalent to owners the important differences between Read More
Redshift create plain user, superuser, createdb user etc…
select * from pg_user_info where usename like ‘mcost%’; create user mcost with createuser password ‘1234Mike’; create Read More
Big Data Blog Posts That I Found Interesting – I am sure there are many others
Best Of Amazon AWS Big Data Blog https://aws.amazon.com/blogs/big-data/orchestrate-multiple-etl-jobs-using-aws-step-functions-and-aws-lambda/ https://github.com/aws-samples/aws-etl-orchestrator https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/best-practices.html#organizingstacks https://aws.amazon.com/blogs/big-data/build-a-data-lake-foundation-with-aws-glue-and-amazon-s3/ https://aws.amazon.com/blogs/big-data/orchestrate-apache-spark-applications-using-aws-step-functions-and-apache-livy/
psql to the rescue – a client program for postgreSQL or Redshift
Here are SOME links to some psql features: http://www.postgresqltutorial.com/psql-commands/ http://www.postgresqltutorial.com/postgresql-list-users/ http://www.postgresqltutorial.com/postgresql-describe-table/ The most current PostgreSQL docs. Read More
Preparing For The Amazon AWS Big Data Specialty Certification Test
Here is some of the material I used to prepare for the Amazon AWS Big Data Read More
installing scala on RHEL 7 or clones and do we really need this with Spark? No!
# you can install this Scala anywhere – but I install it with sudo / root Read More
Hive 2 Connection Woes …
This post is mostly for my reference but, if you are having trouble connecting to a Read More
tcph-kit to generate data
TCPH – http://www.tpc.org/tpch/ Here is where you get the kit to generate data – https://github.com/gregrahn/tpch-kit # Read More
setting up java with java_home jre_home and path
# installing the java jdk via .tar.gz from java.oracle.com to a RHEL / CentOS / Oracle Read More
installing and using hadoop and spark from scratch singlenode and cluster
While this post is a bit backwards (the installation is at the bottom) – it has Read More
Windows 10 Sound / Volume Is Not Consistent
Was driving me crazy – this fixed it. I found an even more simple solution here. The Read More
MongoDB Review / Summary 4.0
MongoDB Review as of Version 4.0 August 2018 MongoDB Overview MongoDB is referred to as a Read More
Using AWS EMR and Spark to Perform ETL
https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-1/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-2-code-development-with-notebooks-and-docker/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-3-running-pyspark-on-emr/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-4-analysing-the-data/ https://www.rittmanmead.com/blog/2016/12/etl-offload-with-spark-and-amazon-emr-part-5/ http://spark.apache.org/docs/latest/index.html
Machine Learning Categorizations and Techniques
Note Ok, I’m obviously new to machine learning – and this post is mostly just for Read More
Redshift data ingestion deduplication / upsert ( delete / insert ) using a staging table – and a little tuning
Why would you do this? In Redshift blocks are immutable – and are re-written completely – Read More