![]() Daemon: Circus: circusd -daemon circus.Font : 120 px Abril Fatface, Helvetica, Arial, Sans-Serif letter-spacing : -5 px Ĭolor : #999 text-shadow : 0 px 3 px 8 px #2a2a2a Ĭolor : #a0a0a0 text-shadow : 0 px 5 px 8 px #2a2a2a įont : 60 px Tahoma, Helvetica, Arial, Sans-Serif Ĭolor : #222 text-shadow : 0 px 2 px 3 px #555.Background, redirect output to logfile: fpython list.py & 2>3 path/to/outfile.log."workon fyp" (name of your virtualenv) to get into the virtualenv.Name the executable as fpython (frameworkpython) Install framework python either by symbolic linking from MAC OS system installed version or follow tutorial here:.Dependencies: pip install -r requirements.txt.Count number of posts between start and end time to arrive at activity frequency.Get username, description, followers, following, location, links to social network, all time views, last 30-day views, profile_image_url, Knows About section.Follow username link to user profile about page, scrap user profile.If most viewed writers section exist, follow link e.g.Look for more topic under related topics section, repeat step 1."quora:topicUrls" -> Set("/sitemap/alphabetical_topics/jp") "so_location" -> "Reading, United Kingdom", "twitter_description" -> "Christian, husband (of father, feminist, software engineer (currently at Google), author, contributor." How to maximize data extraction from Quora (Fast in, fast out).How to ensure new topics are covered? (Alphabetical site map, not sure how often it is updated) How to ensure all topics on Quora are scraped.How to increase overall throughput to process 3 mil users in 7 days (0.2s per user)? Now, it takes ~ 10s per user, 347 days in total.How to overcome IP based throttle limits imposed by SE.How do we determine the activity of each user on a OSN (Activity on Quora, last updated on StackOverflow, ).What if we change the analysis to start from Quora and find matching accounts on Twitter? (Much faster and easier, because we are starting with experts on Quora and after we find the matching twitter account, we can follow Cognos methodology to arrive at expert ranking).How to automate seeding of accounts in order to automatically discover new topics.Using human evaluators - blind testing (BONUS).Tier 1 weighting - proportion of time user spends on each OSN (users/id/network_activity for SO, tweet frequency on Twitter, on Quora)Īnalysis - How did the inclusion of external OSNs i) Change the ranking of experts on Twitter and ii) Improve the system overall Evaluation of System.Part iii - Combining rankings on individual OSNs into overall score ExpertiseRank (variant of PageRank) -, extract User-helped->User graph and run PageRank (BONUS).PageRank - using followee-follower graph of data on Quora, StackOverflow? (BONUS).Ii) Obtain topical similarity between the topic vector and search query using Cover Density Ranking, multiply by log(f) to arrive at final rankings I) Following Cognos, for each user, obtain topic vector where t is set of topics (tags for StackOverflow, topics for Quora on user profile page, inferred from lists for Twitter)Īnd f is i)score for tags for StackOverflow, ii)number of views for topic (use most viewed writers page) on Quora, iii)frequency of occurence of topic in the names and desc of lists containing the user Native - tag-score on StackOverflow, number of views on Quora, ? (no native methods) on Twitter.Part ii - Ranking of experts individually on each OSN for a given topic query Identify foreign accounts using i) Name jaro-wrinkler string-similarity search, ii) Location string matching, iii) Goldberg profile image similarity.Decision of when to stop crawling a non-trivial one which requires investigation.Discover Twitter accounts using a few seed accounts on Twitter Lists, and do a recursive crawl.Link up users from Twitter to their accounts on Quora and StackOverflow. ![]() Methodology Part i - Linking up accounts on different social networks Redis database = v3.2.0 (Compile from source, not brew install!).Identifying topical experts on Twitter using information from StackOverflow and Quora Architecture
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |