The last thing you want is to be flying blind… > Most System Views were not ported! In Snowflake, you can see how much data/how many files were already read, and you can track execution is a neat way.Īfter waiting 1 hour for a COPY command, you will wonder if you gave it more than it could chew. I had copy statements running for more than 90 minutes, and at no point did I know how far along I was, how much was left to run, and if it would end in 2h or 20h. I am loading a gigantic amount of data into Redshift, so I know things will take a bit of time. Flying Blind > No Live Query Execution Tracking Where? Loads of places… let’s get through them one by one. It’s a shame they missed a basic thing though: you can’t really see the outcome of your queries without navigating to detail: I was actually put it on the same level as Snowflake’s, they really did do a good job with it. I have used the web UI for querying it quite a bit and I must confess I liked it a lot. I did not expect it to take that much time, but hey, it was alright. Also, as you would expect with a dataset this big (and this is only a partial…), we had to do some proper encoding/compression. This cardinality is in part due to having attributes in rows instead of columns, but that work was later done inside Redshift. I need to load a dataset which is not very wide, but has an extremely high cardinality. Landing in a role where I had to give it a try, I was actually quite excited about finally putting my hands on it and taking it for a ride. After all, they have the “geek-power”, and they can see the art of doable before putting their fingers to their “mechanical keyboards”. I thought AWS would be in a great position to compete with Snowflake and level the playing field. Knowing how well tech companies leverage each other’s successes (not copying each other of course, just “evolving” ideas), When I heard about Redshift Serverless, I thought “they finally opened their eyes!”.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |