Monday, February 4, 2013

Q: How accurate is Google Flu Trends?

Update March 14, 2013: In 2012–13, Google Flu Trends did not successfully track the target flu indexes in the U.S., France, or Japan. Here are my slides from a talk at the Children's Health Informatics Program (March 14, 2013).

Why this happened is a mystery. Google has said they will present their own view some time this fall. I think the divergence suggests that one needs to be careful about trusting these kinds of machine-generated estimators, even when they work well for three years in a row. It can be hard to predict when they will fall down. (And without an underlying index that is still measured, you might never know when it has stopped working.)

I did an interview with WBUR's CommonHealth blog in January and again in February, and spoke on the radio in January.






Summary: At this point [Feb. 4, 2013], it appears likely that Google Flu Trends has considerably overstated this year's flu activity in the U.S. But we won't be able to draw a firmer conclusion until after the flu season has ended. I don't know why the model broke down this year but am eager to learn, when and if Google comes to a similar conclusion. For now, I suspect this episode may provide a cautionary tale about the limits of inference from "big data" and the perils of overconfidence in a sophisticated and seemingly-omniscient statistical model.
I am not an expert on the flu and you should not make health decisions based on Quora. You should get vaccinated, wash hands often, cover a cough, stay home from work if sick, and follow the CDC's advice: Seasonal Influenza (Flu).

Here are the nationwide figures Google Flu Trends has estimated since its launch, and the underlying CDC index it tries to predict:


The CDC reported Friday (Feb. 1) that for the week of January 13, 2012, through January 19, 2013, 4.5% of doctor visits were by patients with a fever > 100 degrees F and a sore throat or cough (what's known as influenza-like illness). (This was a revision of the original figure of 4.3% that they published the previous Friday before.)

By contrast, on Jan. 20 Google had finalized its prediction for the same statistic: 10.6%. This 6.0-percentage-point difference is larger than has ever occurred before.

For that week, in eight out of the ten HHS regions, Google's predictor was more than double the CDC's figure. (For example, here in New England where I am, Google was saying 14.2% of doctor visits were for influenza-like illness, vs. a CDC figure of only 2.9%.)

Google's scientists quite properly point out that the CDC will adjust its data retroactively, as more data comes in for past weeks. Thus the "red line" on the above graph can change over time, whereas the "blue line" is locked down in all but its very last data point. However, the CDC's adjustments have been modest so far, and even for recent weeks where the Google-CDC divergence is very large, the total number of sites reporting to the CDC has now reached close to the most it ever hits. Further dramatic revision of those data points seems unlikely.

In my nonexpert view, at this point there is little chance that Google Flu Trends's estimates can somehow be vindicated. A similar episode happened in 2009, causing the previous tweak seen above.  Some of their approach may need to change if they want these divergences, repaired only after the fact, to happen less often.

Why the model did not work this year is an exciting mystery, and I am eager to learn the answer, when and if Google reaches a similar conclusion that something went awry.

  • Did one or two of the 160 search queries dramatically increase in popularity? Or was the effect seen over all 160 queries used in the model?
  • Did Google's decision not to retrain the model since 2009 make the difference? If they had retrained the model knowing what they knew in mid-2012, would this season's estimates have been better or worse? Were there hints last summer that, in retrospect, suggest they were mistaken not to retrain the model, and if so can those hints improve the decision process for retraining in the future?
  • Can we evaluate the effect of different retraining policies (e.g. retrain every year, retrain every month vs. the current policy of retraining only on certain triggers), and how they balance various risks? What should the triggers be?
  • Is it possible to estimate flu intensity using queries chosen by a computer without human intervention for their retrospective accuracy? Would the predictions be better if humans intervened to make sure every query made sense as a flu predictor? Or would this simply introduce more problems?

The promise of real-time disease-activity estimates is a valuable idea that has the potential to save lives. Google is the most sophisticated company in the world at this kind of inference, and the fact that even they can apparently stumble suggests that this is really, really tricky. I hope Google improves their technique and continues to attempt it. I'm happy to help them in any way if they could use it, but I doubt they will need me.

===

My graph above may yield a different impression from Google's own graph of their performance, at http://www.google.org/flutrends/about/how.html . That graph looks like this:
Although there is nothing incorrect about this plot, there are a few things that Google could make clearer to reduce the possibility of confusion among readers who have not closely read the scientific papers:

  • The graph shows a "hindcast" from Google's "2009" Flu Trends predictor, the one launched in September 2009 just after the end of the plot. No data point on this graph was actually displayed to the public as a contemporary estimate. From the launch in November 2008 until the end of this plot, Google displayed predictions from the "2008 algorithm." The data from that algorithm is no longer available from Google's site; I had to trace it from a bitmap image that Google submitted in a scientific paper to PLoS ONE about the algorithm's difficulties in 2009 and subsequent improvements (Assessing Google Flu Trends Performance in the United States during the 2009 Influenza Virus A (H1N1) Pandemic).
  • The graph is almost entirely of training data; in other words the CDC data that was used to design the 2009 Google algorithm. It could avoid some confusion if Google were clearer about how much of this graph was actually predicted, versus information that the algorithm was given going in. That would look like this: 
  • The plot ends at the launch of the current algorithm and hasn't been extended to the present. In other words, every single data point in my plot above is missing from Google's plot, and none of the data points on Google's plot are on my plot. In my view, the important thing is the prospective performance of the Google Flu Trends system as it actually estimated the flu. This is the exact opposite of what Google, with its own plot, implicitly expresses it thinks is the relevant figure of merit. (I'm not saying they really believe this; just what the graph says to me.)
  • Although the data shown is from the "2009" algorithm, the only reference Google gives is to their earlier scientific paper about the 2008 algorithm, published in "Nature" (http://research.google.com/archive/papers/detecting-influenza-epidemics.pdf). There is no mention of the later paper or the change in algorithm.
  • In the Nature paper, Google reported that the 2008 algorithm had a 97% mean correlation with the CDC data on a held-out verification set, which is a fantastic result (even higher than the 90% that Google had achieved on their own training set!). In the PLoS ONE paper, published three years later in a less prestigious venue, Google reported that the 2008 algorithm's actual correlation with the first wave of early-2009 "swine flu" was only 29%. I have heard cynics hyperbolize that the purpose of Nature is to publish fantastic-seeming results so that they can be debunked under subsequent scrutiny by less-prestigious journals. It's depressing to see a case where that was somewhat realized.
  • Although Google wrote in the PLoS ONE paper that "We will continue to perform annual updates of Flu Trends models to account for additional changes in behavior, should they occur," and a similar statement in the Nature supplement, in practice Google has not updated the algorithm since September 2009. As they write below, they determined that an update wasn't necessary. But they could make this more clear -- as well as discussing how they determine whether or not to update the model -- in their papers and Web site.

The fact that Google decided not to update the model for 2012-13, and subsequently the model performed poorly in 2012-13, suggests that the procedure for deciding when an update is necessary may need to be reworked. On the other hand, it's possible that even if Google had updated the model, the divergence would have been just as bad (or worse). The difference may mean different things for how Google Flu Trends can be improved in the future. These are questions I sincerely hope Google examines and answers in a future scientific paper or Web site update.

315 comments:

  1. Thank you for the useful article. It has helped a lot in training my students. Keep writing more.
    best big data training in Chennai

    ReplyDelete
  2. The expansion of internet and intelligence in business process lead the way to huge volume of data. It is important to maintain and process these data to be efficient in data handling. Hadoop Training in Chennai | Big Data Training in Chennai

    ReplyDelete
  3. Welcome to Wiztech Automation - Embedded System Training in Chennai. We have knowledgeable Team for Embedded Courses handling and we also are after Job Placements offer provide once your Successful Completion of Course. We are Providing on Microcontrollers such as 8051, PIC, AVR, ARM7, ARM9, ARM11 and RTOS. Free Accommodation, Individual Focus, Best Lab facilities, 100% Practical Training and Job opportunities.

    Embedded System Training in chennai
    Embedded System Training Institute in chennai
    Embedded Training in chennai
    Embedded Course in chennai
    Best Embedded System Training in chennai
    Best Embedded System Training Institute in chennai
    Best Embedded System Training Institutes in chennai
    Embedded Training Institute in chennai
    Embedded System Course in chennai
    Best Embedded System Training in chennai

    ReplyDelete
  4. Paris airport transfer - Parisairportransfer is very common in Paris that provides facilities to both the businessmen and the tourists. We provide airport transfers from London to any airport in London and also cruise transfer services at very affordable price to our valuable clients.

    Paris taxi
    Paris airport shuttle
    paris hotel transfer
    paris airport transfer
    paris shuttle
    paris car service
    paris airport service
    disneyland paris transfer
    paris airport transportation
    beauvais airport transfer
    taxi beauvais airport
    taxi cdg airport
    taxi orly airport

    ReplyDelete
  5. Buzz Applications is offering website for your product or your company!!

    ReplyDelete
  6. I am following your blog from the beginning, it was so distinct & I had a chance to collect conglomeration of information that helps me a lot to improvise myself.
    Regards,
    CCNA Training in Chennai | CCNA Training Institute in Chennai | Best CCNA Training in Chennai

    ReplyDelete
  7. That is it! You are currently sending SMS upgrades!SMS Marketing Applications On the off chance that you need to change settings and alter the SMS messages that are sent, keep perusing.

    ReplyDelete
  8. I learned some new information. Thanks for sharing.

    Magento Development in Chennai

    ReplyDelete
  9. My Arcus offer java training with 100% placement. Our java training course that includes fundamentals and advance java training program with high priority jobs. java j2ee training with placement having more exposure in most of the industry nowadays in depth manner of java

    java training in chennai

    ReplyDelete

  10. useful information

    Online recruitment for bank jobs and government jobs and you can get Notification and application to apply online for bank jobs and govt jobs

    ReplyDelete
  11. GK Updates (GK), Latest Current Affairs, General knowledge, Current Affairs, Daily Current Affairs, GK, Todays GK, Daily GK Updates,This Year Latest Current Affairs, India Current Affairs GK, General, Knowledge, Quiz, Awareness, Questions, Answers, Explanation, Interview, Entrance, GK,Exams.

    ReplyDelete
  12. Useful Information……
    Recruitment voice contains Daily GK Updates, Bank Recruitment, Government jobs, Bank jobs, Interview Tips, Banking News, GK Updates and Latest Jobs. @ http://recruitmentvoice.com

    ReplyDelete
  13. Great article. Keep sharing such a informative post.

    web designing institute in chennai

    ReplyDelete
  14. Nice Information :
    Telugu Cinema Contains Telugu Cinema News, Latest Movie Reviews, Actor, Actress, Movie Galleries And Many More @ http://www.telugucinema.com/

    ReplyDelete
  15. Great article. Glad to find your blog. Thanks for sharing.

    web designing training in chennai

    ReplyDelete
  16. Nice Information:
    Telugu Cinema Contains Telugu Cinema News, Latest Movie Reviews, Actor, Actress, Movie Galleries And Many More Telugu Cinema News

    ReplyDelete
  17. Nice Information
    one and only affiliate agency in south INDIA, earn money online from affiliate network in india

    ReplyDelete
  18. Java Corporate Training Companies in Chennai Corporate Training Companies in India Corporate Training Companies in India

    Corporate Training in Chennai Corporate Training in Chennai

    Corporate Training Companies in India Corporate Training Companies in India

    Java Corporate Training Companies in India Java Corporate Training Companies in India

    ReplyDelete
  19. The Google Apps Marketplace is another online store for incorporated business applications that permits Google Apps clients to effortlessly find You can now customize your Google .com with a foundation picture. https://800support.net/google-latest-news/googles-new-launch/

    ReplyDelete
  20. this is very use full information.....and thanks for sharing....
    qlikview Training in Chennai



    ReplyDelete

  21. Our Complete in depth Java training course takes you to TOP Level IT companies with high end package. Arcus Offers Java J2EE real time training with placement assurance..
    java training in chennai

    ReplyDelete
  22. Thank you so much for sharing this useful information. ETL Testing Online Training is excellent solution for learning from home MSBI Online Training

    ReplyDelete
  23. I use this service https://www.intistele.com/blog/get-started-with-sms-marketing/ when I need to address several issues that are important to keep in touch with the client, not to lose it. Often I am using discounts on birthday. A good tip to stimulate demand.

    ReplyDelete
  24. it is very usefull inforamation thanq..telugubullet.com also a super site for latest telugu news updates.

    ReplyDelete
  25. If you are really looking for a genuine advice and suggestion for Yahoo issues, you can dial a Yahoo Customer Care to have any kind of suggestion. Their suggestions and advices are free, so you can find any time with their toll-free number.

    ReplyDelete
  26. The yahoo mail account is one of the oldest email service there is and it is not very difficult to handle the yahoo mail related troubles that you face when you have proper knowledge of how to handle any technical issues that do crop up while you are working with the account.

    ReplyDelete
  27. an article in Science magazine titled The Parable of Google Flu: Traps in Big Data Analysis exposed Flu Trends’ use of faulty data collection methods for the 2012-2013 flu season. The article called the program “big data hubris,” for overestimating doctor visits for the flu compared with CDC reports.

    “Quantity of data does not mean that one can ignore foundational issues of measurement and construct validity and reliability and dependencies among data. The core challenge is that most big data that have received popular attention are not the output of instruments designed to produce valid and reliable data amenable for scientific analysis,” the authors of the Science magazine wrote, which led to dozens of media reports questioning the relevancy of Flu Trends.

    http://kosmiktechnologies.com/msbi/

    ReplyDelete
  28. Thanks for sharing the Informative article.
    http://kosmiktechnologies.com/msbi/

    ReplyDelete
  29. Awesome post.
    I read your blog everything is helpful and effective.
    Thanks for sharing with us.
    Java Training in Chennai

    ReplyDelete
  30. Nice Post keep updating like this,

    telugu 2015 movies

    ReplyDelete
  31. Nice information. It looks very informative informations to us. I have to forward your blog to my friends. keep sharing. Thank you. Hadoop Training in Chennai | Android Training in Chennai | Python Training in Chennai

    ReplyDelete
  32. Gmail provides great customer service to help its customers solve all types of Gmail related problems. The technical help at Gmail provide best solutions to users for variety of problems that they face from time to time. The technical problems create trouble in using the Gmail account smoothly and this disrupts the work of the Gmail users.

    ReplyDelete
  33. Your blog is very interest about the Google Flu Trends, Perfectprofile.net is a leading Job Portal in india. if you want etl testing jobs in hyderabad please go through this website.

    ReplyDelete
  34. There is a technique to save email messages in the occasion a record is hacked. Two Internet Gmail Technical Support Contact Number accounts with separate and extraordinary passwords are used to go down all messages. In the occasion one record is hacked, the other record saves the email.

    ReplyDelete
  35. This comment has been removed by the author.

    ReplyDelete
  36. really you have shared very informative post. the post will be helpful to many peoples. thank you for sharing this blog. so keep on sharing such kind of an interesting article.
    python training in chennai

    ReplyDelete
  37. I understand the trends of google flu trends.This blog is very useful to me.The information you provide is very nice.Thanks for sharing.


    SAS Training in Bangalore

    ReplyDelete
  38. Thanks for this valuable information.I was really learn about content.If you want walk-ins in hyderabad for etl testing fresher’s.check this site.

    ReplyDelete

  39. This is excellent information. It is amazing and wonderful to visit your site.Thanks for sharng this information,this is useful to me...
    Android Training in Chennai
    Ios Training in Chennai

    ReplyDelete
  40. In short you would always get help for any type of trouble from the Gmail technical support team and the number is available toll free globally around the world.
    The Gmail customer service number is available round the clock for any type of help that you want. The number is global and is easily available round the clock from any place

    ReplyDelete
  41. Have you ever undergone the ordeal of waiting for the technical support from the yahoo tech support engineers? Such things are not the characteristic features of yahoo customer service. Yahoo customer service is not the newly born component of yahoo .From the very beginning, yahoo customer service has been very much integral to yahoo services. From time to time, certain changes and improvements have been introduced to yahoo customer service with a view to meet the changing needs of the yahoo users. So, if you are a yahoo user, you are certainly fortunate enough to have the yahoo support professionals always available at your closest disposal.

    ReplyDelete
  42. In case you are not able to give the permanent fix to the particular issue, you need to immediately dial Gmail customer support 1-800 phone number for live chat. The phone number will give the quick response for the particular trouble in your Gmail account. Just get in touch with Gmail customer care experts and obtain the splendid solution.

    ReplyDelete
  43. Hello I am so delighted I located your I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work.

    ReplyDelete
  44. In case you are using Gmail account for quite a longer period of time and confront an issue in downloading emails on your smart device, you don’t need to get worried at all. For the particular trouble, you need to call Gmail customer care 800 telephone number anytime.

    ReplyDelete
  45. I really enjoy simply reading all of your weblogs. Simply wanted to inform you that you have people like me who appreciate your work. Thanks ..
    Python Training in Chennai | Big Data Analytics Training in Chennai

    ReplyDelete
  46. Role of education, its failer to deliver equal oppurtutiny to women.http://www.how-todo.xyz/

    ReplyDelete
  47. In case if your Yahoo email account is not working responding properly and you are finding for instant resolution then you can rely on us. Simply pickup your phone and dial Yahoo customer service helpline number which is toll-free.

    ReplyDelete
  48. "Forgot account password" is a very common issue which almost every email users have faced once in the lifetime. If you forgot password for your Yahoo mail account, you can easily retrieve your password by using the Yahoo password assistance page.

    ReplyDelete
  49. Hi, I have read your blog. Really very informative and excellent post I had ever seen about Python. Thank you for sharing such a wonderful blog to our vision. Learn Python Training in Chennai to know more details about this technology.
    Web Designing Training in Chennai|Big Data Analytics Training in Chennai

    ReplyDelete
  50. This information useful who search for  CCNA jobs in Hyderabad  .So we need this information.Thank you for sharing this great post.

    ReplyDelete
  51. Hi admin.., I read your blog completely.I Gather lot of information in this blog that information will help me develop my knowledge. Thanks for sharing. keep sharing more blogs.


    Core Java Online Training

    ReplyDelete
  52. Skype is very useful and power full software and it will be good for personal or official work. You can use Skype for texting, sharing data, audio call, video call, video conferencing and much more or If you want skype setup in your window 10. Than this is your guide. If you face any problem don’t worries! contact with us and visit our website us they are available 24*7 to help.

    ReplyDelete
  53. Skype application is well known for its variety of quality features. One of the most interesting features that Skype offers is Group chats. It allows making group video chats with up to 25 persons at a time.Our experts can be contacted via our toll-free number, email and live chat as well. Moreover, our communication lines are open 24 hours. So, feel free to get in touch with Skype Support Team or Skype Live Chat . We assure to serve you with utmost quality services.

    ReplyDelete
  54. If you're unable to merge your Pages, it means that your Pages aren't eligible to be merged. And if you don’t have any solution to facebook page merge so doesn’t worry we help you to solve this problem visit our website or contact with contact service team.

    ReplyDelete
  55. It was a wonderful chance to visit this kind of site and I am happy to know. thank you so much for giving us a chance to have this opportunity! I will be back soon for updates.click here

    ReplyDelete
  56. I have read your blog and I gathered some needful information from your blog. Keep update your blog. Awaiting for your next update.
    SAS Online Training
    R Programming Online Training|
    Tableau Online Training|

    ReplyDelete
  57. Really fantastic. you are really a skilled blogger.I like your post very much.Thanks for sharing.keep sharing more blog.

    White Label Website Builder

    ReplyDelete
  58. Hats off to your presence of mind..I really enjoyed reading your blog. I really appreciate your information which you shared with us.

    Hadoop Training in Marathahalli|
    Hadoop Training in Bangalore|
    Data science training in Marathahalli|
    Data science training in Bangalore|

    ReplyDelete
  59. Its a fabulous blog i have ever read in past days. search jobs near by you in chennai.

    ReplyDelete
  60. This is one of the main things you can do and also one of the most important things you can do. google dashboard

    ReplyDelete
  61. I simply wanted to write down a quick word to say thanks to you for those wonderful tips and hints you are showing on this site.
    hadoop training in chennai

    ReplyDelete
  62. Your new valuable key points imply much a person like me and extremely more to my office workers. With thanks; from every one of us.

    selenium training in chennai|

    ReplyDelete
  63. Needed to compose you a very little word to thank you yet again regarding the nice suggestions you’ve contributed here.

    Java Training in Bangalore

    ReplyDelete
  64. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all the details.
    http://www.traininginmarathahalli.in/amazon-web-services-training-in-bangalore/

    ReplyDelete
  65. Thanks for one marvelous posting! I enjoyed reading it; you are a great author. I will make sure to bookmark your blog and may come back someday. I want to encourage that you continue your great posts, have a nice weekend!


    Java Training in Chennai


    Java Training in Bangalore


    Java Training in Bangalore

    ReplyDelete
  66. Thanks for everything. Anybody who wants support for Facebook related issues, call at Facebook contact number 1 (407) 965-2136.

    ReplyDelete
  67. Wow, I'm really impressed by this website, not only does it look good, the posts are super interesting too! Subscribed! By the way, if you need hep with singapore company incorporation, let us help, we are the best business incorporation assistant. register a business immediately!

    ReplyDelete
  68. This comment has been removed by the author.

    ReplyDelete
  69. All of Facebook password reset issues can be solved with us at 8663243042. Visit http://www.facebookhelp.us

    ReplyDelete
  70. Informative post about hadoop, i am looking forward for realtime hadoop online training institute.

    ReplyDelete
  71. This comment has been removed by the author.

    ReplyDelete
  72. I like the post format as you create user engagement in the complete article. It seems round up of all published posts. Thanks for gauging the informative posts.
    cara menggugurkan kandungan
    http://jualobatpenggugurkandungan.net/ obat penggugur kandungan
    http://tandatandakehamilan.net/ tanda tanda kehamilan
    http://tandatandakehamilan.net/cara-cepat-dan-selamat-menggugurkan-kandungan/ cara menggugurkan kandungan
    http://obataborsi59.com/ obat aborsi
    http://obataborsi59.com/cara-menggugurkan-kandungan-dengan-cepat-dan-aman/ cara menggugurkan kandungan

    ReplyDelete

  73. It has been simply incredibly generous with you to provide openly what exactly many individuals would’ve marketed for an eBook to end up making some cash for their end, primarily given that you could have tried it in the event you wanted.

    https://www.besanttechnologies.com/training-courses/data-science-training-in-bangalore


    https://www.besanttechnologies.com/training-courses/data-warehousing-training/datascience-training-institute-in-chennai

    ReplyDelete
  74. Your good knowledge and kindness in playing with all the pieces were very useful. I don’t know what I would have done if I had not encountered such a step like this...Hadoop Training in Chennai

    ReplyDelete
  75. Awesome website with awesome articles keeps up the good work.Online Coursework Help

    ReplyDelete
  76. This blog is very helpful for beginners and experts also, thanks for sharing it. Keep share content on MSBI Online Training Hyderabad

    ReplyDelete
  77. Excellent article. Very interesting to read. I really love to read such a nice article. Thanks! keep rocking.
    Seo Companies in bangalore
    low cost seo plans
    Online Marketing Services in bangalore

    ReplyDelete
  78. Thanks for your information sharing with us, We are waiting for your more updates on MSBI Online Training Hyderabad

    ReplyDelete
  79. I found your this post while searching for some related information on blog search...Its a good post..keep posting and update the information. promote website

    ReplyDelete
  80. Thank You for sharing your article, This is an interesting & informative blog. It is very useful for the developer like me. Kindly keep blogging. Besides that Wisen has established as Best Corporate Training in Chennai .

    Nowadays JavaScript has tons of job opportunities on various vertical industry. Know more about JavaScript Framework Training visit Corporate Training Companies in India.

    This post gives me detailed information about the technology. corporate training companies in india

    ReplyDelete
  81. The beauty of SEO is the fact once your site and even blog benefits ground, the benefit the idea purchased from the search engine optimisation surgery is pretty much long term. Employing advertising, you need to preserve paying for your website website visitors; with search engine marketing, you get more and more targeted traffic plus it gets increasingly less expensive because website's standing increases.Blog Comment

    ReplyDelete
  82. Hi There,


    Thank you! Thank you! Thank you! Your blog was a total game changer!


    If you have any idea about that and you have the secure and efective way to send files through socket I'll appreciate that.
    This the code that I am using, it works well for the small files, but for the big ones don't work

    Great effort, I wish I saw it earlier. Would have saved my day :)


    Gracias
    Uday

    ReplyDelete
  83. Thanks for your valuable answer. I clearly understand about google flu trends with your answer. Thank you.

    Best PHP Training in Chennai | Best PHP Training Institute in Chennai

    ReplyDelete
  84. why would you know anything about how Google marketing lists your website, social media, pay-per-click, YouTube and others can play a critical role in your business? https://edkentmedia.com/ppc-management-toronto/

    ReplyDelete

  85. I believe there are many more pleasurable opportunities ahead for individuals that looked at your site.

    Selenium Training in Porur

    ReplyDelete
  86. I feel really happy to have seen your webpage and look forward to so many more entertaining times reading here. Thanks once more for all
    the details.

    white label website builder

    ReplyDelete
  87. Well Said, you have furnished the right information that will be useful to anyone at all time. Thanks for sharing your Ideas.
    Magento course in chennai

    ReplyDelete
  88. I wish to show thanks to you just for bailing me out of this particular trouble.As a result of checking through the net and meeting techniques that were not productive, I thought my life was done. JAVA Training in chennai

    ReplyDelete
  89. I like your all post. You have done really good work. Thank you for the information you provide, it helped me a lot. I hope to have many more entries or so from you.
    Very interesting blog.
    Cara Menggugurkan kandungan

    Obat Aborsi

    obat aborsi

    cara menggugurkan kandungan

    obat aborsi

    cara menggugurkan kandungan

    cara menggugurkan kandungan

    obat aborsi

    obat penggugur kandungan

    ReplyDelete
  90. This comment has been removed by the author.

    ReplyDelete
  91. Your posts is really helpful for me.Thanks for your wonderful post. I am very happy to read your post. It is really very helpful for us and I have gathered some important information from this blog.
    Java Training in Chennai | Java Training Institute in Chennai

    ReplyDelete
  92. This was an nice and amazing and the given contents were very useful and the precision has given here is good.
    RPA Training in Chennai

    ReplyDelete
  93. I found your this post while searching for information about blog-related research ... It's a good post .. keep posting and updating information. login to hotmail

    ReplyDelete
  94. This was an nice and amazing and the given contents were very useful and the precision has given here is good.
    Java Training in Chennai

    ReplyDelete
  95. usefull article about "How accurate is Google Flu Trends". thanks for sharing.
    Data Science Training in Chennai

    ReplyDelete
  96. excellent topics gathered lots of informations,keep share your post


    Hadoop Training in Chennai

    ReplyDelete

  97. Hello. This post couldn’t be written any better! Reading this post reminds me of my previous roommate. He always kept chatting about this. I will forward this page to him.
    bigdata training institute in bangalore

    ReplyDelete
  98. Thanks for the wonderful information were given here and thanks for sharing.
    AWS Training in Chennai

    ReplyDelete
  99. Wonderful bloggers like yourself who would positively reply encouraged me to be more open and engaging in commenting.So know it's helpful.

    java training in annanagar | java training in chennai

    java training in marathahalli | java training in btm layout

    java training in rajaji nagar | java training in jayanagar

    ReplyDelete
  100. This comment has been removed by the author.

    ReplyDelete
  101. Amazing blog about the various informative information on the programming languages... Java Training in Chennai | Pega Training in Chennai

    ReplyDelete
  102. Thanks Admin for sharing such a useful post, I hope it’s useful to many individuals for developing their skill to get good career.
    python online training
    python training course in chennai
    python training in jayanagar

    ReplyDelete
  103. Awesome post. Really you are shared very informative concept... Thank you for sharing. Keep on updating...

    Guest posting sites
    Education

    ReplyDelete
  104. The blog which you are shared is very much helpful for us to knew about the web designing. thanks for your information.
    Web Designing Institute
    Best Web Design Courses
    Web Design Training Courses
    Learn Website Design
    Best Way to Learn Web Design

    ReplyDelete
  105. Read all the information that i've given in above article. It'll give u the whole idea about it.
    Devops Training in pune

    ReplyDelete
  106. This is a good post. This post give truly quality information. I’m definitely going to look into it. Really very useful tips are provided here. thank you so much. Keep up the good works.
    RPA Training in Chennai
    Robotics Process Automation Training in Chennai
    RPA courses in Chennai
    Robotic Process Automation Training
    RPA course

    ReplyDelete
  107. thank you for sharing such a nice and interesting blog with us. i have seen that all will say the same thing repeatedly. But in your blog, I had a chance to get some useful and unique information. I would like to suggest your blog in my dude circle. please keep on updates. hope it might be much useful for us. keep on updating...
    German Classes in Chennai
    German Language Classes in Chennai
    German Language Course in Chennai
    German Courses in Chennai
    Best german classes in chennai
    German language training in chennai

    ReplyDelete
  108. Nice and good article. It is very useful for me to learn and understand easily. Thanks for sharing your valuable information and time. Please keep updating.
    Web Design Training

    ReplyDelete
  109. It has been just unfathomably liberal with you to give straightforwardly what precisely numerous people would've promoted for an eBook to wind up making some money for their end, basically given that you could have attempted it in the occasion you needed.nebosh course in chennai

    ReplyDelete
  110. Innovative thinking of you in this blog makes me very useful to learn.
    i need more info to learn so kindly update it.
    Java Institutes in bangalore
    Java Courses in Thirumangalam
    Java Training Institutes in Numgambakkam
    Java Training in Karapakkam

    ReplyDelete
  111. I am reading your post from the beginning, it was so interesting to read & I feel thanks to you for posting such a good blog, keep updates regularly.
    nebosh course in chennai

    ReplyDelete
  112. Really very nice blog information for this one and more technical skills are improve,i like that kind of post....

    selenium training in chennai

    java training in chennai

    ReplyDelete
  113. Good job in presenting the correct content with the clear explanation. The content looks real with valid information. Good Work

    DevOps is currently a popular model currently organizations all over the world moving towards to it. Your post gave a clear idea about knowing the DevOps model and its importance.

    Good to learn about DevOps at this time.


    devops training in chennai | devops training in chennai with placement | devops training in chennai omr | devops training in velachery | devops training in chennai tambaram | devops institutes in chennai | devops certification in chennai | trending technologies list 2018

    ReplyDelete
  114. Hi, Your post is quite great to view and easy way to grab the extra knowledge. Thank you for your share with us. I like to visit your site again for my future reference.
    Java Training in Chennai
    PHP Training in Chennai
    Core Java Training in Chennai
    Java Training center in Chennai
    PHP Training
    Best PHP training in chennai

    ReplyDelete
  115. Thanks for sharing such a great information.Its really nice and informative great content of different kinds of the valuable information's.The style of writing is excellent and also the content is top-notch. Thanks for that shrewdness you provide the readers!

    Oracle DBA training in Chennai
    Oracle DBA Training

    ReplyDelete

  116. Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article.thank you for sharing such a great blog with us. expecting for your.
    Selenium training in chennai
    Selenium training institute in Chennai
    iOS Course Chennai
    Digital Marketing Training in Chennai
    hp loadrunner training
    Loadrunner Training in Velachery

    ReplyDelete
  117. Great blog!!! It was very impressed to me. I like so much and keep sharing. Thank you.

    Guest posting sites
    Technology

    ReplyDelete
  118. Thank you for the useful article. It has helped a lot. top college in Jaipur

    ReplyDelete
  119. Nice and good article. It is very useful for me to learn and understand easily.
    The best college in Jaipur

    ReplyDelete

  120. The blog which you have shared is more useful for us. Thanks for your information. Waiting for your more updates
    German Language Course
    German Courses in Coimbatore
    German Courses Near Me
    Learn German Course
    German Language Training

    ReplyDelete
  121. Excellent Blog!!! Such an interesting blog with clear vision, this will definitely help for beginner to make them update.
    Android Training in Chennai
    Selenium Training in Chennai
    Digital Marketing Training in Chennai
    JAVA Training in Chennai

    ReplyDelete
  122. Thanks for sharing this interesting blog with us.My pleasure to being here on your blog..I wanna come beck here for new post from your site.

    Qtp training in Chennai
    Android Training in Chennai
    Selenium Training in Chennai
    Digital Marketing Training in Chennai
    JAVA Training in Chennai

    ReplyDelete
  123. This information is impressive. I am inspired with your post writing style & how continuously you describe this topic. Eagerly waiting for your new blog keep doing more.
    German Classes in Chennai
    Digital Marketing Training in Chennai
    JAVA Training in Chennai
    Selenium Course in Chennai
    selenium testing training in chennai

    ReplyDelete
  124. Thanks for sharing this unique information with us. Your post is really awesome. Your blog is really helpful for me..
    organic oil
    organic oil in jaipur
    organic cold pressed oils
    ayurvedic oil store in jaipur

    ReplyDelete
  125. Thanks for sharing this unique information with us. Your post is really awesome. Your blog is really helpful for me..
    organic oil
    organic oil in jaipur
    organic cold pressed oils
    ayurvedic oil store in jaipur

    ReplyDelete
  126. Hello! Someone in my Facebook group shared this website with us, so I came to give it a look. I’m enjoying the information. I’m bookmarking and will be tweeting this to my followers! Wonderful blog and amazing design and style.
    safety course in chennai

    ReplyDelete
  127. Try playing online gambling on the BGAOC website. great blackjack online Only here are permanent and Bosnian wins.

    ReplyDelete
  128. Hmm, it seems like your site ate my first comment (it was extremely long) so I guess I’ll just sum it up what I had written and say, I’m thoroughly enjoying your blog. I as well as an aspiring blog writer, but I’m still new to the whole thing. Do you have any recommendations for newbie blog writers? I’d appreciate it.
    Advanced AWS Training in Bangalore | Best Amazon Web Services Training Institute in Bangalore
    Advanced AWS Training Institute in Pune | Best Amazon Web Services Training Institute in Pune
    Advanced AWS Online Training Institute in india | Best Online AWS Certification Course in india
    AWS training in bangalore | Best aws training in bangalore

    ReplyDelete
  129. Very nice post here thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
    machine learning with python course in chennai
    machine learning course in chennai
    best training insitute for machine learning
    Android training in velachery
    PMP training in chennai

    ReplyDelete
  130. Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's. 
    microsoft azure training in bangalore
    rpa training in bangalore
    best rpa training in bangalore
    rpa online training

    ReplyDelete
  131. I really want to thank you for such kind of wonderful blog posted for us.I expect this kind of updation soon in your page.
    Software testing training in chennai
    Salesforce Training in Chennai
    Android Training in Chennai
    Digital Marketing Training in Chennai

    ReplyDelete

Note: Only a member of this blog may post a comment.