User:Lucia Dossin/Protyping/Assignment 5 - Revision history

Lucia Dossin at 19:41, 31 March 2014

2014-03-31T19:41:05Z

@@ Line 95: / Line 95: @@
 </source>
 <b>4.</b>Searches and displays on browser<br>
-<source lang="cgi">
+<source lang="python">
 </source>

Lucia Dossin at 19:39, 31 March 2014

2014-03-31T19:39:58Z

← Older revision		Revision as of 21:39, 31 March 2014
Line 93:		Line 93:

	writer.commit()		writer.commit()
			</source>
			<b>4.</b>Searches and displays on browser<br>
			<source lang="cgi">
	</source>		</source>

Lucia Dossin at 21:24, 29 March 2014

2014-03-29T21:24:01Z

← Older revision		Revision as of 23:24, 29 March 2014
Line 28:		Line 28:

	<h2>Screenshots</h2>		<h2>Screenshots</h2>
			One of the first versions.<br>
			[[File:Lucia_Fai-small.ogv]]

	<h2>Code</h2>		<h2>Code</h2>

Lucia Dossin at 20:47, 29 March 2014

2014-03-29T20:47:52Z

← Older revision		Revision as of 22:47, 29 March 2014
Line 21:		Line 21:
	There is still a lot to explore in this project: for example, investigating/changing the parameters which control the score of each word, recording the clicks and the association path generated in each visit (possiblly displaying it to the user), adding new indexes (in the same fashion of Google's Images, News, Videos tabs, there could be several indexes where the same word would display different results, according to the 'nature' of the index).		There is still a lot to explore in this project: for example, investigating/changing the parameters which control the score of each word, recording the clicks and the association path generated in each visit (possiblly displaying it to the user), adding new indexes (in the same fashion of Google's Images, News, Videos tabs, there could be several indexes where the same word would display different results, according to the 'nature' of the index).

	Also, some improvements could be done, such as allowing the search of more than one word, and building a more interesting index, with less pronouns and other non-meaningful words, for example.		Also, some improvements could be done, such as allowing the search of more than one word, and building a more interesting index, with less pronouns and other non-meaningful words, and remove or unify duplicate entries for example. There are certainly more improvements to be done, these are only a few examples.

	Besides that, I would also like to allow the user to suggest a correlation to the term being searched, so that the results would display both the 'internal (official)' correlations as well as 'external' ones.		Besides that, I would also like to allow the user to suggest a correlation to the term being searched, so that the results would display both the 'internal (official)' correlations as well as 'external' ones.

Lucia Dossin at 20:43, 29 March 2014

2014-03-29T20:43:51Z

← Older revision		Revision as of 22:43, 29 March 2014
Line 31:		Line 31:
	<h2>Code</h2>		<h2>Code</h2>
	<b>1.</b>Reads the txt files and writes new ones, with one word in each line.<br>		<b>1.</b>Reads the txt files and writes new ones, with one word in each line.<br>
	This is ~~done~~ twice: ~~for~~ the titles and ~~for~~ the content.		This is code was run twice: to generate the titles and the content.
	<source lang="python">		<source lang="python">
	#!/usr/bin/env python		#!/usr/bin/env python

Lucia Dossin at 20:41, 29 March 2014

2014-03-29T20:41:32Z

← Older revision		Revision as of 22:41, 29 March 2014
Line 56:		Line 56:
	<br>		<br>
	<b>3.</b>Adds documents to the index<br>		<b>3.</b>Adds documents to the index<br>
	<source lang="python>		<source lang="python">
	from whoosh.fields import Schema, TEXT, KEYWORD		from whoosh.fields import Schema, TEXT, KEYWORD
	from whoosh.index import open_dir		from whoosh.index import open_dir

Lucia Dossin at 20:41, 29 March 2014

2014-03-29T20:41:00Z

← Older revision		Revision as of 22:41, 29 March 2014
Line 55:		Line 55:
	</source>		</source>
	<br>		<br>
			<b>3.</b>Adds documents to the index<br>
			<source lang="python>
			from whoosh.fields import Schema, TEXT, KEYWORD
			from whoosh.index import open_dir
			import random
			import math

			ix = open_dir("index")

			cont=[]
			removed=['to', 'and', 'is', 'for', 'from', 'the', 'a', 'an', 'of', 'in', 'at', 'with', 'this', 'that', 'or', 'by']

			with open("initial_content/out-content.txt") as c:
			for line in c:
			if line != ' ' and line != '\n':
			line = line[:-1]
			if line not in removed:
			cont.append(str(line))

			writer = ix.writer()

			with open("initial_content/out-titles.txt") as f:
			for line in f:
			if line != ' ' and line != '\n':
			line = line[:-1]
			line = line.lower()
			if line not in removed:
			r=[]
			for n in range(9):
			r.append( str(cont[random.randint(1,len(cont)-1)])))
			s = line
			for lin in range(9):
			writer.update_document(title=u'"'+ s + '"', content=u'"'+ r[lin] +'"')
			writer.add_document(title=u'"'+ r[3] + '"', content=u'"'+ s +'"')

			writer.commit()
			</source>

Lucia Dossin at 20:37, 29 March 2014

2014-03-29T20:37:58Z

← Older revision		Revision as of 22:37, 29 March 2014
Line 40:		Line 40:
	out.write(w+'\n')		out.write(w+'\n')
	</source>		</source>
			<br>
			<b>2.</b>Creates an index<br>
			<source lang="python">
			import os.path
			from whoosh.index import create_in
			from whoosh.index import open_dir
			from whoosh.fields import Schema, STORED, ID, KEYWORD, TEXT

			schema = Schema(title=TEXT(stored=True), content=TEXT(stored=True), tags=TEXT(stored=True ))

			if not os.path.exists("index"):
			os.mkdir("index")
			ix = create_in("index", schema)
			</source>
			<br>

Lucia Dossin at 20:33, 29 March 2014

2014-03-29T20:33:50Z

← Older revision		Revision as of 22:33, 29 March 2014
Line 30:		Line 30:

	<h2>Code</h2>		<h2>Code</h2>
	1.Reads the txt files and writes new ones, with one word in each line.<br>		<b>1.</b>Reads the txt files and writes new ones, with one word in each line.<br>
	This is done twice: for the titles and for the content.		This is done twice: for the titles and for the content.
	<source lang="python">		<source lang="python">

Lucia Dossin at 20:33, 29 March 2014

2014-03-29T20:33:23Z

← Older revision		Revision as of 22:33, 29 March 2014
Line 35:		Line 35:
	#!/usr/bin/env python		#!/usr/bin/env python
	with open("initial_content/titles-file.txt") as c, open('out-titles.txt', 'w') as out:		with open("initial_content/titles-file.txt") as c, open('out-titles.txt', 'w') as out:
	for ~~num,~~ line in ~~enumerate(~~c~~, 1)~~:		for line in c:
	~~print 'line number:', num~~
	~~print line~~
	words = line.split()		words = line.split()
	for w in words:		for w in words:
	out.write(w+'\n'		out.write(w+'\n')
	</source>		</source>