App Engine is great but if you are using Docker, you can only use Flexible Environment which is not what I want as I would love to continue to use Standard Environment. Here is a comparison of the two environments.
My goal is to use Docker for development only. When the app is deployed, it will still be deployed to App Engine Standard Environment as usual. After some researches and trial, I found a way.
Recently I finished watching a documentatory, The Veitnam War, on Netflix. This is one of the best war documentatory I have ever watched, I have to say. It covers both how the decisions were made in the governments, as well as the experience of soliders and civilians from all sides.
BigQuery date partitioned tables can limit the data scan by partitions to help keep the query cost low and improve query performance. However, Google's documents do not give much clue how to use partitioned tables to create views that support partition queries. I did some research and found the way to do it, even with views that are created from joining partitioned tables.
Update: After posting this article, a friend from Facebook pointed out that the list of advertisers are companies provided personal info to match my Facebook info to run targeting campaigns, instead of Facebook giving away the contact info to these companies. However, I see companies I have never heard of and they are not in Australia from some quick Google searches. If that's the case, how these companies got my email address or some kind of identifier in the first place is truly worrying.
Facebook sells your data to advertisers to run marketing campaigns targeting you, that's no news. Finally, a company called Cambridge Analytica abused it enough to make the headlines last week. This article gives a good summary of what's going on: EVERYTHING YOU NEED TO KNOW ABOUT FACEBOOK AND CAMBRIDGE ANALYTICA
I am wondering how many advertisers actually have my data over these years through Facebook. It turns out that my contact info was sold to 69 companies by Facebook. If you are wondering how many advertisers have your contact info, here is the way to find out in just few minutes.
I am a big fan of history, and I read a lot about history. The reason I think today could be a very important day in history is because of this news. You might think that's no big deal, we are far away from China and its politics. I bet a lot of people had the same thought in 44BC.
Update: Just to clarify, Insightech is built in a totally different architecture and it is way more complex than this.
I gave a presentation in MeasureCamp Melbourne, and the topic was "Build Your Own Web Analytics Platform in 25 Minutes". In MeasureCamp, each session has 25 minutes presentation time, that's where the 25 minutes in the topic from. What inspired me to do this talk was a conversation in Measure Slack about sending Google Analytics data into your own database. And I think it is an interesting thing to do to build a web analytics platform from scratch without spending a lot of time and money.
Building your own web analytics platform can be much simpler than you would think. In this presentation, there is pretty much no coding knowledge required. If you are familiar with AWS S3 and CloudFront, it is very simple. If not, don't worry, AWS has really good interface so you just need to click through a series of buttons.
You may already notice that these days, after you look at a product on a website, you start to see the product in ads everywhere. This is called remarketing. It is a very common marketing technique being used to sell a product that you are probably interested in, or even already purchased. To me, if you show me ads about a product I recently purchased, it is most likely that you are wasting money. Advertisers could suspend this kind of ads by setting up a negative audience group that are customers who already purchased, so the product won't show to the customers in any ads for a certain period of time (e.g. 30 days). However, it seems most advertisers don't bother to do it, especially retailer websites, I guess maybe the setup cost to suspend ads based on purchases is probably bigger than the money they could save. I also heard that saving advertising costs is not in the interest of marketing because that means the budget will be smaller next year. I am interested to know if anyone actually did any analysis or research on this.
Recently, I experience a pretty extreme case of poor ads experience that it's not only a waste of advertiser's money, it's also a waste of my time and it really frustrated me. I am interested in the Logitech K400 Plus keyboard, and Bing Lee was doing a promotion for $29. That is way lower than the normal retail price $49 ~ $59. So I went to Bing Lee website to try to buy it, but it was out of stock. In the next few days, I see the ads about this keyboard.
Back in mid 2015, I created my first and only Python script so far. The script was a voice assistant. I thought I lost the source code, but it's been in my Dropbox all the time and was named with a weird project name so I didn't realised that's it. I just didn't try to look for it hard enough.
Here are a couple videos I took back in 2015 when it was built. The script was running on a Raspberry Pi 3 with a USB sound card connecting to a microphone and a speaker.
Today is my last day at IAG, the place I worked in the last 5 years. Initially I was planning to work there for 6 months as a contractor to help them enhance the web analytics capability, and then to start to do my own thing. But it turns out I stayed there for 5 years, contracting for almost 2.5 years and then I became a permanent employee for a bit more than 2.5 years. It was a good place to work with a lot of nice people. I left because I have been building a web analytics time machine and I have to focus on it now.
A couple years ago I started to think about how to make web analytics implementation easier and faster, so that we as analysts can focus on insights and generating true value for businesses. That's how I started to create the concept of "web analytics time machine". The idea is to build a system that can re-create the moment visitors use the website, so analysts can do analysis like travelling back in time to investigate and identify the barriers to conversion. This time machine won't let you to change the history, but you can use it to set the future.
I moved my website chaoming.li to weebly again. I actually had a self hosted WordPress blog but it's just too much trouble to maintain a WordPress blog when moving things to EC2. So this is a fresh start, and I will post some of the old articles to this new website later.
One of the reasons I picked weebly was its Android mobile app. I would like to see if it helps to keep the blog updated. Time will tell.