Friday, November 5, 2010

Protest in Polish Parliament Against Senior Chinese Official Accused of Genocide

A screen shot from Polish TVP24 of the two protestors being escorted by a parliamentary security guard. (Screenshot of TVP24)
WARSAW, Poland—A senior Chinese government official accused in Spain of genocide and torture against Falun Gong practitioners in China, was confronted by human rights protesters in the halls of the Polish Parliament in Warsaw Thursday morning.

"We wanted to meet the delegation face-to-face and get a message out to Mr. Jia Qingling: that the persecution of Falun Gong in China that he is co-responsible for is wrong and must be stopped", said Tomasz Kowalski, president of the Polish Falun Dafa Association and one of the protesters.

See Epochtimes details http://www.theepochtimes.com/n2/content/view/45448/

Tuesday, November 2, 2010

Kate Middleton's parents spend weekend at Queen's estate

Amid rumors that a royal engagement is imminent, Kate Middleton's parents were spotted weekending at Queen Elizabeth's Balmoral estate in Scotland.
Michael and Carole Middleton spent the weekend with their daughter and Prince William at Birkhall, Prince Charles’s private residence on the Balmoral estate.

The move is seen as highly symbolic; former aides suggested that the middle-class Middleton family were now firmly welcomed into the royal fold, reports the Daily Mail. "It’s just a matter of time," one aid told the paper.
Kate’s parents were photographed receiving instructions in shooting. "This is the Middletons being taught to be able to join the royals at play," according to the Daily Mail's source.

Brazil's First Woman President Sets Her Agenda

RIO DE JANEIRO, Brazil—In her first televised speech since wining the presidential election, Dilma Rousseff, the next head of state of Brazil, praised current President Luiz Inácio Lula da Silva and emphasized women's rights. She also promised to respect freedom of religion, freedom of the press, and human rights, but above all, she will work to eradicate poverty, reported O Globo.

“We cannot rest while there are hungry Brazilians and while poor children are abandoned to their own luck,” said Rousseff. “It is about an abyss in a developing Brazil.”

More details

The Marketing Corner: Black Friday Countdown

Procrastination can sometimes work in your favor—that is, if you are a consumer.

As a small business owner, procrastination is not your friend. If you sat around and felt you had all the time to put your plan together for Black Friday, like sand in an hour glass, your time is just about come and gone.

More details

Monday, November 1, 2010

Indonesia’s Mount Merapi Erupts Again

Indonesia’s Mount Merapi erupted again Monday, spitting smoke, ash and hot gasses, AP reported. So far no injuries have been reported.

Merapi, the most active of Indonesia’s volcanoes, has erupted several times since Tuesday Oct. 26, killing 38 and forcing the evacuation of 40,000 residents to temporary shelters.

“There are more and more refugees coming down the mountain,” Rukman, the head of disaster response for the Indonesian Red Cross (PMI), told the Christian Science Monitor. “We need water and better sanitation; that is the big problem.”

See detail

Sunday, October 31, 2010

Indonesia Tsunami Death Toll Rises to over 400

The tsunami which swept across Indonesia’s northern Mentawai Islands has now claimed at least 413 lives, reports the Jakarta Post, with 298 still missing.

The tsunami, triggered by a 7.7-magnitude undersea earthquake on Oct. 25, destroyed six of 27 villages on islands and displaced 4000 people.

See Details

Chinese Public Warned to Expect Further Inflation

T
hough inflation exists in China, rarely will an official warn the public to expect it to continue.

A member of the Chinese National Development and Reform Commission recently spoke anonymously to domestic media, saying that further interest rate increases will not control inflation so the public must simply be resigned to it, according to Deutsche Welle.

Subsequently, many media outlets, including China's state-run Xinhua News Agency, reported numerous responses from Internet users. The public decried the NDRC’s warning as illogical.
See more details

US Govt. Investigating China Trade

WASHINGTON—The United Steelworkers union (USW) opposed China’s entrance into the WTO ten years ago, fearing the loss of American manufacturing jobs. Today, the steelworkers are still fighting, saying that by not adhering to WTO rules, China gets an unfair advantage over the United States.

“We fought China’s accession to the WTO because we know that they cheat and they lie and they say we want to be players and play by the rules, and they don’t,” says Gary Hubbard, a spokesperson for United Steelworkers.

In September the U.S. Trade Representative accepted a 5,800-page petition from the USW accusing China's clean energy sector of wide-spread violations of WTO rules.
See more detail

Friday, October 29, 2010

New York Real Estate Expo Set to Inspire Industry

NEW YORK—Distress, lending, wealth, green building, new rules… the list of topics covered at the upcoming NYC Network Real Estate Expo reads as an “all you need to know about real estate in New York.”

From Robert Knakal’s talk on the midterm elections and “How They May Impact Commercial Real Estate Values,” to William Procida’s distressed property seminar, the Mariott Marquis in Times Square will be buzzing on Nov. 5.

Many real estate professionals are gearing up for the event, with organizers anticipating up to 4,000 attendees, double the figure from last year.

See more

China’s ‘Map World’ Without Google

hina’s State Bureau of Surveying and Mapping (SBSM) announced that it will redistribute the market of Internet maps at the end of this year or early next year. Noticeably, Google Maps is not included.

On Oct. 21, China’s Map World, a national, public Internet service of geographic information, was officially up and running. The new Map World has almost the same functions as Google Maps. Information includes roads, water systems, administrative districts, residential areas, and geographic names. It has real-image mode and three-dimensional mode. It also provides information on business locations, such as shops and restaurants. According to the SBSM, Map World will be updated every six months. So far, this website cannot provide high-resolution images of areas outside China.

In March, Google withdrew its Internet search service from China and moved its servers to Hong Kong to avoid Chinese regime censorship. Since then Google’s Internet service market share has been dropping in China. However, mainland Internet users can still access Google, including its map service, through the Hong Kong server.
Chinese Business License

In the past, map information in China has been classified as military secret. And survey and mapping services are still restricted in China. A company must obtain a license before it can provide these services. To apply for the license, the company server that keeps the mapping information must be located inside China, and the company must provide the server’s public IP.

In May, China issued a new regulation called and said all companies who provide Internet geographic information must apply for business permits.

The that were announced in 2007, stipulate that foreign businesses involved in mapping in China, must have joint venture and cooperation with a Chinese business. Also, the Chinese company must be the holding company.

As of the end of September, authorities have issued business permits to 31 companies, including Tencent, Baidu, Sina, and Nokia. Google is not among them.

Since Google Maps did not get a license to operate in China, it has no chance of entering the map service market in China, unless Google moves its servers back to China and agrees to let a Chinese company hold the majority of its stock.
Related Articles

* The Seduction of China’s Red Carpet
* China’s Censorship Shakes the Foundation of Google’s Search

Same Satellite Images

After China set up Map World, Chinese media said that the software is the intellectual property of the Chinese government. However, a Chinese Internet user found out that the satellite images on Map World and Google Maps are exactly the same in many instances, including the number of cars on highways, the cars’ shapes, positions, and colors. The Internet user surmised that the two satellite maps have to come from the same source.

Jiang Jie, Chief of the Bureau’s Center of National Basic Geological Information, admitted, “the satellite images are purchased from overseas.”

So far Google has not commented.

Tuesday, March 23, 2010

Resolve around File Descriptor limited on Reactor Select (selector)

I've been running into problem file descriptor run out using the default reactor. By default, twisted got hardcode the file descriptor to be 1024 (MAX). If you want to change the default MAX need to modify code and recompile. Or we can use the pollreactor to increase several thousand more.
>>
from twisted.internet import pollreactor
pollreactor.install()
<<
usage as normal:
from twisted.internet import reactor

Even if you change the ulimit on filesystem, it doesn't change with the library. This is something hardcode inside twisted lib.

Creating a self-signed cert with one command

Creating a self-signed cert with one command

The following command will generate a new key and create a certificate all in one line suitable for use by Apache or any other SSL tool.

openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout www.example.com.pem -out www.example.com.pem

You can then use the file above in apache with the following two lines

SSLEngine On
SSLCertificateFile www.example.com.pem

Monday, March 22, 2010

python twisted interrupt signal handler


from twisted.internet import reactor
import signal
def shutdown(*f):
sig, ob = f
if sig == signal.SIGINT: reactor.stop() #shutdown the reactor if Ctrl-C enter.

signal.signal(signal.SIGINT, shutdown)
reator.run(installSignalHandlers=0)

twisted.web2 auto reconnecting browser

Making use web2 Broker Class from http://code.google.com/p/twisted-web2-client

class Browser:
#Init Channel
#Authenticate Channel
#Send Channel
#Recieve Channel
def __init__(self, timeout=180 ):
self.timeout = timeout
self.conn = Broker()
self.headers = {}
self.response_headers = None

def parse_response_headers(self, headers):
self.response_headers = headers
self.raw_headers = headers._raw_headers
if headers.hasHeader('set-cookie'):
parseSetCookies = http_headers.parseSetCookie(headers.getRawHeaders('set-cookie'))
self.headers['Cookie']= parseSetCookies

def getResponseBody(self, resp):
d = defer.Deferred()
self.parse_response_headers(resp.headers)
stack = []
stream_mod.readStream(resp.stream, stack.append).addCallback(lambda x: ''.join(stack)).addCallbacks(d.callback, d.errback)
return d

def connect(self, uri, postdata=None, method='GET', *args, **kwargs):
d = self.conn.ask(uri, method, self.headers, postdata,timeout=self.timeout, *args, **kwargs).addCallback(self.getResponseBody)
return d

Monday, March 8, 2010

httpclient for twisted web2

Here is excellent tutorial for twisted web2 client.
http://code.google.com/p/twisted-web2-client/

Thanks to the author, I've spend quite sometime to google, but no luck!

Friday, March 5, 2010

install and config weblizer on ubuntu

http://fontignie.blogspot.com/2006/04/install-and-configure-webalizer-on.html

Install and configure webalizer on Ubuntu

  1. install webalizer
    sudo apt-get install webalizer
    If webalizer is not found, you have to add sources in /etc/apt/sources.list: can comment the universe sources.

  2. Enable the apache2 hostname resolution: go into /etc/apache2/apache2.conf: Change
    HostnameLookups Off
    into
    HostnameLookups On

  3. By default webalizer is not well configured: it does not check in the good log. In /etc/webalizer.conf: Change

    LogFile /var/log/apache2/access.log.1
    to
    LogFile         /var/log/apache2/access.log

  4. Test webalizer:

    sudo webalizer

    If you get a warning like warning: Truncating ...
    It is because you did not put the hostnameLookup. If afterward, you get this message, it can be because you are attacked by a virus...

  5. Run webalizer as a cronjob. This has to be run as root: Edit the root cronjobs by running the command:
    sudo crontab -e
    and add the line:
    0 * * * * webalizer
    With that line, every hour webalizer is run

install and config weblizer on ubuntu

http://fontignie.blogspot.com/2006/04/install-and-configure-webalizer-on.html

Install and configure webalizer on Ubuntu

  1. install webalizer
    sudo apt-get install webalizer
    If webalizer is not found, you have to add sources in /etc/apt/sources.list: can comment the universe sources.

  2. Enable the apache2 hostname resolution: go into /etc/apache2/apache2.conf: Change
    HostnameLookups Off
    into
    HostnameLookups On

  3. By default webalizer is not well configured: it does not check in the good log. In /etc/webalizer.conf: Change

    LogFile /var/log/apache2/access.log.1
    to
    LogFile         /var/log/apache2/access.log

  4. Test webalizer:

    sudo webalizer

    If you get a warning like warning: Truncating ...
    It is because you did not put the hostnameLookup. If afterward, you get this message, it can be because you are attacked by a virus...

  5. Run webalizer as a cronjob. This has to be run as root: Edit the root cronjobs by running the command:
    sudo crontab -e
    and add the line:
    0 * * * * webalizer
    With that line, every hour webalizer is run

squid log analysis

http://www.safesquid.com/html/viewtopic.php?t=2395
http://cord.de/tools/squid/calamaris/
http://www.squid-cache.org/Scripts/

Thursday, March 4, 2010

Good Learning material for twisted and asynchronous programming

http://krondo.com/blog/?p=1209

Excellent documentation

Wednesday, March 3, 2010

Configure network ubuntu/redhat

Redhat:
manually eth0: edit /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=00:e0:81:26:1e:9e
ONBOOT=no
BOOTPROTO=none
TYPE=Ethernet
NETMASK=255.255.255.0
IPADDR=192.168.1.214
GATEWAY=192.168.1.1

Using Commands: None-Gui command (system-config-network-tui)
GUI command: system-config-network &
Reference : http://www.cyberciti.biz/faq/rhel-centos-fedoracore-linux-network-card-configuration/

Ubuntu:
edit file: /etc/network/interfaces
iface eth0 inet static
address 192.168.1.100
netmask 255.255.255.0
network 192.168.1.0
broadcast 192.168.1.255
gateway 192.168.1.254

Restart the network.
sudo /etc/init.d/networking restart

Reference: http://www.cyberciti.biz/tips/howto-ubuntu-linux-convert-dhcp-network-configuration-to-static-ip-configuration.html

Monday, March 1, 2010

created a defered function

def Normal(args):
def startMessages():
for key in sendMSGSequence:
self.deferQueue.put(key)
sleep(1) # simulate like a real GUI serial Connection.
#print key
reactor.callLater(1, startMessages)

The inner function will execute after 1 second callback from the reactor function. The Reactor is continue is execution with being block.

Wednesday, February 24, 2010

Gương người xưa: Thực hành nhân đức và để lại tiếng thơm

Trương Phương Bình, sống ở Nam Kinh dưới thời Bắc Tống, là người khoan dung nhân hậu và trọng lễ nghĩa. Ông luôn giữ vững tiết tháo cao thượng và rất tin tưởng vào Thần Phật. Là người có phong cách cao nhã và độ lượng, ông luôn được nhiều người kính trọng.

Vào mùa thu năm 1054 sau Công nguyên, tức năm Chí Hòa thứ nhất đời vua Tống Nhân Tông, có một tin đồn ở vùng Tứ Xuyên truyền đến rằng quân địch đang sắp sửa xâm phạm biên giới. Quân sĩ đóng nơi biên cương tỉnh Tứ Xuyên nửa đêm được tin vô cùng kinh hãi, trăm họ đều tháo chạy, đạo tặc nổi lên khắp nơi, trật tự xã hội trở nên đại loạn.

Khi tin tức truyền đến kinh thành ở Khai Phong, cả triều đình từ trên xuống dưới đều chấn động kinh hãi. Tống Nhân Tông phải tìm người chủ soái để phái đi dẹp loạn, bèn nói với triều thần: “Đừng gây nên họa loạn, cũng đừng khiến nó trở thành sự biến. Dẫu rằng tin đồn trong dân chúng có tới tấp đi nữa, nhưng chủ ý của Trẫm đã định, họa xâm lăng chưa chắc là sẽ đến, chỉ sợ sự biến nổi lên từ nội bộ mà thôi. Sự việc này cần giải quyết theo cả hai cách là ‘nhu văn’ và ‘võ công’. Trẫm cần một hoặc hai vị đại thần đi xử lý chuyện này một cách khéo léo. Ai có khả năng làm nổi sự việc này, Trẫm sẽ phái người đó đi chiêu an quân dân của Trẫm”. Mọi người trong triều đều tiến cử Trương Phương Bình. Tống Nhân Tông đồng ý và phái Trương Công đi. More

From: Dai Phat Thanh Hy Vong (vietsoh.com)

Petitioners Beware: Detainment, Detainment, Then ‘Reeducation Through Labor’

By Rona Rui
[Theepochtimes.com] Epoch Times Staff
Created: Feb 23, 2010 Last Updated: Feb 23, 2010

This billboard reads 'Persistent and Noisy Petitioners: First time—detention, second time—detention, third time—Reeducation Through Labor,' set up by 'The Chinese Communist Party Tianxin Township Committee, Tianxin Township People's Government.' (Photo by anonymous citizen)
A billboard with a foreboding message erected beside a road in Jiahe County, China’s Hunan Province, is indicative of the ruling regime’s tighter control of dissent, according to commentators: “To All Persistent and Noisy Petitioners: First time—Detention; Second time—Detention; Third time—Reeducation Through Labor.”

According to an eyewitness, this sign is located in Tianxin Township in Jiahe County to the right of a highway. Standing in a fork in the road, the sign stood at 7-8 square meters and was inscribed with “The Chinese Communist Party (CCP) Tianxin Township Committee, Tianxin Township People's Government.” Photos of forced labor detainees, large handcuffs, and policemen with austere expressions made it a startling sight, according to an observer. More

Tuesday, February 23, 2010

Jack in the Box offering free sandwich -- today only

At your local Jack in the Box you can buy a large soda and received a FREE grilled sandwich. The offer, available today only, is an introduction to the two new items at Jack in the Box.

The sandwiches, the turkey bacon cheddar and deli trio sandwiches, are grilled and are said to be made with fresh ingredients. Since the yummy items won't cost you a dime (and sound great for lunch or dinner) there is nothing wrong with stopping by and giving in to an inexpensive meal.

There are a few places in the country this offer is not valid (almost everywhere though.) Check your local Jack in the Box or if you want to save some time, got to the Jack in the Box website to see if your area is a freebie location.

Monday, February 22, 2010

Twisted's DeferredQueue

import itertools
from twisted.internet import defer
from twisted.internet import reactor
from twisted.internet import task

FREQUENCY = 10

def startProducing(q):
"""Put an incrementing number in the queue every 1/FREQUENCY
seconds.
"""
counter = itertools.count()
def produce():
q.put(counter.next())
task.LoopingCall(produce).start(1.0/FREQUENCY)

def startConsuming(q):
"""Consume an object from the queue and print it.
"""
def consumer(o):
print o
q.get().addCallback(consumer)
q.get().addCallback(consumer)

q = defer.DeferredQueue()
startProducing(q)
startConsuming(q)

reactor.run()

Saturday, February 20, 2010

China’s Crises in Tibet and Xinjiang Beijing believes investment is the way out

Source: Theepochtimes.com

While those in charge of propaganda work to intoxicate the country with the image of a “harmonious society” at the behest of party leader Hu Jintao, frequent uprisings in China’s western ethnic provinces of Tibet and Xinjiang have, to the embarrassment of officials, laid bare the lie.

In fact, the intensifying social conflicts in the west have pushed China’s teetering stability to the limit.

Official Chinese media reported that a Tibetan protest on March 14, 2008 resulted in 13 deaths and an economic loss of 200 million yuan (US$29 million). Over the following nine months, numerous other protests broke out in Tibet and surrounding areas. ...more ...

Friday, February 19, 2010

create Defered function from non-asynchronous blocking code

def largeFibonnaciNumber():
"""
Represent a long running blocking function by calculating
the TARGETth Fibonnaci number
"""
TARGET = 10000

first = 0
second = 1

for i in xrange(TARGET - 1):
new = first + second
first = second
second = new

return second

from twisted.internet import threads, reactor

def fibonacciCallback(result):
"""
Callback which manages the largeFibonnaciNumber result by
printing it out
"""
print "largeFibonnaciNumber result =", result
# make sure the reactor stops after the callback chain finishes,
# just so that this example terminates
reactor.stop()

def run():
"""
Run a series of operations, deferring the largeFibonnaciNumber
operation to a thread and performing some other operations after
adding the callback
"""
# get our Deferred which will be called with the largeFibonnaciNumber result
d = threads.deferToThread(largeFibonnaciNumber)
# add our callback to print it out
d.addCallback(fibonacciCallback)
print "1st line after the addition of the callback"
print "2nd line after the addition of the callback"

if __name__ == '__main__':
run()
reactor.run()

python DOM example

http://docs.python.org/library/xml.dom.minidom.html
xml.dom.minidom — Lightweight DOM implementation

Deferred Reference



This document is a guide to the behaviour of the twisted.internet.defer.Deferred object, and to various ways you can use them when they are returned by functions.

This document assumes that you are familiar with the basic principle that the Twisted framework is structured around: asynchronous, callback-based programming, where instead of having blocking code in your program or using threads to run blocking code, you have functions that return immediately and then begin a callback chain when data is available.

After reading this document, the reader should expect to be able to deal with most simple APIs in Twisted and Twisted-using code that return Deferreds.

* what sorts of things you can do when you get a Deferred from a function call; and
* how you can write your code to robustly handle errors in Deferred code

Deferreds are beautiful! (A Tutorial)

Introduction

Deferreds are quite possibly the single most confusing topic that a newcomer to Twisted has to deal with. I am going to forgo the normal talk about what deferreds are, what they aren't, and why they're used in Twisted. Instead, I'm going show you the logic behind what they do.

A deferred allows you to encapsulate the logic that you'd normally use to make a series of function calls after receiving a result into a single object. In the examples that follow, I'll first show you what's going to go on behind the scenes in the deferred chain, then show you the deferred API calls that set up that chain. All of these examples are runnable code, so feel free to play around with them.

Linux System Administration and Configuration

http://www.yolinux.com/TUTORIALS/LinuxTutorialSysAdmin.html#PROCESSES

Generating Deferreds

twistmatrix python doc

Deferred objects are signals that a function you have called does not yet have the data you want available. When a function returns a Deferred object, your calling function attaches callbacks to it to handle the data when available.

This document addresses the other half of the question: writing functions that return Deferreds, that is, constructing Deferred objects, arranging for them to be returned immediately without blocking until data is available, and firing their callbacks when the data is available.

This document assumes that you are familiar with the asynchronous model used by Twisted, and with using deferreds returned by functions .

Using the Twisted Web Client

http://twistedmatrix.com/documents/current/web/howto/client.html

This document describes how to use the HTTP client included in Twisted Web. After reading it, you should be able to make HTTP requests using Twisted Web. You will be able to specify the request method, headers, and body and you will be able to retrieve the response code, headers, and body.

Limiting Parallelism

Concurrency can be a great way to speed things up, but what happens when you have too much concurrency? Overloading a system or a network can be detrimental to performance. Often there is a peak in performance at a particular level of concurrency. Executing a particular number of tasks in parallel will be easier than ever with Twisted 2.5 and Python 2.5:

from twisted.internet import defer, task

def parallel(iterable, count, callable, *args, **named):
coop = task.Cooperator()
work = (callable(elem, *args, **named) for elem in iterable)
return defer.DeferredList([coop.coiterate(work) for i in xrange(count)])

Here's an example of using this to save the contents of a bunch of URLs which are listed one per line in a text file, downloading at most fifty at a time:

from twisted.python import log
from twisted.internet import reactor
from twisted.web import client

def download((url, fileName)):
return client.downloadPage(url, file(fileName, 'wb'))

urls = [(url, str(n)) for (n, url) in enumerate(file('urls.txt'))]
finished = parallel(urls, 50, download)
finished.addErrback(log.err)
finished.addCallback(lambda ign: reactor.stop())

Source

python twisted Throttling with Cooperator

Example 8: Throttling with Cooperator


from twisted.internet import reactor
from twisted.web.client import getPage
from twisted.internet import defer, task

maxRun = 2

urls = [
'http://twistedmatrix.com',
'http://twistedsoftwarefoundation.org',
'http://yahoo.com',
'http://www.google.com',
]

def pageCallback(result):
print len(result)
return result

def doWork():
for url in urls:
d = getPage(url)
d.addCallback(pageCallback)
yield d

def finish(ign):
reactor.stop()

def test():
deferreds = []
coop = task.Cooperator()
work = doWork()
for i in xrange(maxRun):
d = coop.coiterate(work)
deferreds.append(d)
dl = defer.DeferredList(deferreds)
dl.addCallback(finish)

test()
reactor.run()

This is the last example for this post, and it's is probably the most arcane :-) This example is taken from JP's blog post from a couple years ago. Our observation in the previous example about the way that the deferreds were created in the for loop and how they were run is now our counter example. What if we want to limit when the deferreds are created? What if we're using deferred semaphore to create 1000 deferreds (but only running them 50 at a time), but running out of file descriptors? Cooperator to the rescue.

This one is going to require a little more explanation :-) Let's see if we can move through the justifications for the strangeness clearly:

1. We need the deferreds to be yielded so that the callback is not created until it's actually needed (as opposed to the situation in the deferred semaphore example where all the deferreds were created at once).
2. We need to call doWork before the for loop so that the generator is created outside the loop. thus making our way through the URLs (calling it inside the loop would give us all four URLs every iteration).
3. We removed the result-processing callback on the deferred list because coop.coiterate swallows our results; if we need to process, we have to do it with pageCallback.
4. We still use a deferred list as the means to determine when all the batches have finished.

This example could have been written much more concisely: the doWork function could have been left in test as a generator expression and test's for loop could have been a list comprehension. However, the point is to show very clearly what is going on.

I hope these examples were informative and provide some practical insight on working with deferreds in your Twisted projects :-)
Source

twisted task and schedule handler

from twisted.internet import task, reactor
import time
def timer():
print time.ctime()
loop = task.LoopingCall(timer)
loop.start(1, now=True)
reactor.run()

python urllib2 handle cookies

To handle python cookies in urllib2, it's already provide with new urllib2.

Here is a example to deal with cookies return from http server.
opener = urllib2.build_opener(HTTPCookieProcessor(),HTTPBasicAuthHandler(),HTTPDefaultErrorHandler( ))
opener.addheaders = [('User-agent', 'Mozilla/5.0'),("Content-type", "text/xml; charset=\"UTF-8\"")]
opener.open(url)
urllib2.install_opener(opener)
urllib2.open(new_url)

We have register the opener into urllib2, afterward; we can use urllib2.open() without worry about handling cookies return.

python conversion unicode to ascii

unicodeStr = something
ascii = unicodeStr.encode('ascii','ignore')

Usage Python twisted

For new page usage:
from twisted.internet import reactor
from twisted.internet.defer import DeferredList

data = "some data for http body"

def responseData(result, factory)
print "Content Length=", len(result)
print "Cookies =", factory.cookies

def failurePage(error)
print 'msg=', error.getErrorMessage()
print 'err=', error

def finished(ign):
reactor.stop()

d1, factory = getNewPage(url, postdata=data)

dl = DeferredList([d1])
dl.addCallback(responseData, factory)
dl.addErrback(failurePage)
dl.addCallback(finished)

reactor.run()

Đức Đạt Lai Lạt Ma viếng thăm Hoa Kỳ

Đức Đạt Lai Lạt Ma viếng thăm Hoa Kỳ
Đăng ngày: 19-02-2010 lúc 5:26 PM Font size:



Đức Đạt Lai Lạt Ma đã đến Washington hôm thứ Tư, trước một cuộc họp với Tổng thống Hoa Kỳ Barack Obama.

Chế độ cộng sản Trung Quốc đã thúc giục chính quyền của tổng thống Obama hủy bỏ cuộc họp. Họ coi Đức Đạt Lai Lạt Ma như một phần tử ly khai.

Lưu tâm tới sự nhạy cảm của chế độ Trung Quốc, ông Obama đã trì hoãn cuộc họp với Đức Đạt Lai Lạt Ma cho đến sau khi ông gặp những người lãnh đạo Trung Quốc trước trong chuyến công du Châu Á tháng 11 năm ngoái.

Như một cử chỉ nhượng bộ nữa trước Trung Quốc, ông Obama sẽ không tiếp Đức Đạt Lai Lạt Ma tại Phòng Bầu dục của Nhà trắng. Thay vào đó, ông sẽ đón tiếp Ngài ở Phòng Bản đồ, nơi thường được dùng cho các cuộc gặp gỡ mang tính xã hội.

Tin tức Tân Đường Nhân, ngày 18 tháng 02 năm 2010.

Tags : Tin nổi bật
Chuyên mục: Tin thế giới
from: vietsoh.com

twisted framework

Modified getPage () to return a factory and deferred, so that we when extract the return cookies.


from twisted.web import client
def getNewPage(url, contextFactory=None, *args, **kwargs):
"""
Download a web page as a string and return clientfactory.
Download a page. Return a deferred, which will callback with a
page (as a string) or errback with a description of the error.

See HTTPClientFactory to see what extra args can be passed.
"""
factory = client._makeGetterFactory(url,client.HTTPClientFactory, contextFactory=contextFactory, *args, **kwargs)
return factory.deferred, factory

Tuesday, January 5, 2010

The Year of Ponzi Schemes Ushers In Tighter Regulation

http://www.theepochtimes.com/n2/content/view/27439/

Financier and fraudster Bernie Madoff was the worst but far from the only Ponzi purveyor nabbed in 2009. (Chris Hondros/Getty Images)
NEW YORK—2009 may go down in the annals of business notoriety as the “Year of Ponzi schemes.”

In early 2009, Bernie Madoff’s $65 billion Ponzi scheme unraveled to a world in disbelief. The former Chairman of the NASDAQ stock exchange in March pleaded guilty to defrauding investors and turning his asset management business Bernard L. Madoff Investment Securities into one giant Ponzi scheme.

In total, the $65 billion in fabricated gains were the largest investor fraud in history, and Madoff is currently in prison serving a 150-year sentence.

But that’s not all. Sir Allen Stanford, a Texas billionaire, was arrested last summer on charges of inventing an $8 billion pyramid scheme. His Stanford International Bank, an offshore holding company based in Antigua, allegedly sold more than $7 billion in CDs to unsuspecting depositors, promising sky-high interest rates.

In addition, Stanford will stand trial for bribing Antiguan finance officials.

Another prominent fraud involves Tom Petters, CEO of the Minneapolis-based Petters Group Worldwide LLC. Petters was convicted on Dec. 2 of carrying out a $3.65 billion Ponzi scheme.

Petters was guilty of wire and mail fraud, as well as money laundering. According to reports, Petters falsified documents to obtain huge loans to support his scam and his venture, which employed more than 2,000 people.

House of Cards

So why were so many frauds uncovered in 2009? The answer lies in that Ponzi schemes require new investors to fund old ones, and as the global recession hit, more investors than usual sought to withdraw their funds.

During boom years, investors have little use for their savings. But during a recession as money is drawn and no new investors can be found, Ponzi schemes are often uncovered.

"I think this has been an incredibly difficult year," former U.S. Securities and Exchange Commision (SEC) Chairman Harvey Pitt told Reuters Television. "We've seen a lot of financial scandals, a great deal of misconduct, an economic meltdown and a concerted push now for meaningful regulatory reform."

In total, the SEC filed 664 civil complaints in 2009 against alleged fraudsters. In 2009, more than 150 Ponzi schemes collapsed, compared to about 40 in 2008, according to the AP’s examination of court records and FBI filings.

And that will heighten regulators’ vigilance against fraud schemes in 2010.

On Dec. 16, 2009, the SEC installed new rules for money managers—investment managers will require surprise inspections by the SEC. The ruling comes more than a year after Madoff admitted to his sons that his business is “one big lie.”

Mary Schapiro, current Chair of the SEC, said in a December meeting that the new guidelines came about from “the Madoff Ponzi scheme and other frauds in which investor assets were misappropriated by investment advisers. Such frauds have caused investors to question whether their assets are safe when they entrust them to an investment adviser.”

Previously, the SEC was criticized for not reining in Madoff earlier, despite several leads and tips that the SEC received but ultimately dismissed in prior years.




Saturday, January 2, 2010

Buying and Selling Thesis Papers Common in China

http://www.theepochtimes.com/n2/content/view/27315/

The buying and selling of academic thesis papers has become so common that in 2007 alone the sales revenue was between 180 million and 540 million yuan (S$26 million to $79 million), according to Chinese media reports.


Buyers from universities and research institutes purchase the theses they need over the internet, according to Associate Professor Shen Yang from Wuhan University School of Information Management, quoted in a Yangtze Daily report.

'Mindboggling' corruption

On Dec. 17, Sound of Hope radio interviewed two professors in China to find out more about the practice.

Dr. Sun, a retired professor from Shandong University, said it’s one of the ways that corruption in academia manifests. “Students, professors, and researchers—they are all involved. Students need to have their papers published in order to graduate,” he said. Professors and scientists need published papers to obtain promotions that reflect the status of their titles, and some Communist Party officials need an academic degree to gain a promotion, Sun explained, all of which drives the market. “The range of needs dictates the phenomenon of publication sales—a multi-million dollar business in Chinese society.”

It shows the corruption of Chinese society, Sun continued. “Plagiarism has always existed in China. Directors of academic departments are often politicians with power, but they lack an academic background. When they push for a promotion, besides bribing members of review boards, they pay for people to write research papers for them,” he said. With a loose review process for research papers, an ample sales market for them has formed. “The extent of the corruption within academic institutions is mindboggling.”

Dr. Jiao Guobiao, associate professor of Beijing University’s Department of Journalism, says that the evaluation process for the promotion of teachers, as well as prospective graduating students, all depends on whether they have published papers.

Jiao was blacklisted in China, meaning his name cannot appear in any Chinese media reports, after he published a scathing critique of the role of the Central Propaganda Department in 2004.

“China has rigid standards for who should write what kind of papers and under which circumstances. For students who also need to work part-time, it’s evident that they have neither the time nor the resources to write papers,” he said “They will end up buying their semester, year, or graduation papers. I know this is what happens.”

The Ministry of Education in China apparently plays at least a tacit role in the paper sales scheme. “The Ministry of Education or the universities only expect you to meet the requirement for the number of papers; their expectations are not based on whether you write them or not,” he said. “As long as you have a paper to meet the evaluation criteria, everyone is happy.”

Thousands in New Year Hong Kong March for Democracy

Pro-democracy demonstators march in Hong Kong on January 1, 2010 calling for universal sufferage and the release of political prisoners.
Pro-democracy demonstators march in Hong Kong on January 1, 2010 calling for universal sufferage and the release of political prisoners. ( Anthony Dickson/AFP/Getty Images)
[www.theepochtimes.com] HONG KONG - Thousands of Hong Kong residents appealed to China on New Year's Day to allow full democracy to be introduced soon in the city, as opposition lawmakers pressed forward with a mass resignation plan later this month.

Congregating outside the city's historic domed legislature, protesters carried colourful banners with slogans such as "Democracy Now!" and made their way to Beijing's representative office.

Some demonstrators held aloft portraits of Chinese dissident Liu Xiaobo, demanding the release of the prominent activist and writer, jailed last week for 11 years on a subversion charge.

Organisers said more than 30,000 protesters turned out for the New Year's Day "return our right to universal suffrage" march. Police put the number at around 9,000.

Hundreds of police erected steel barricades as protesters with loudhailers converged on Beijing's liaison office in the former British colony, returned to Chinese rule in 1997.

There were minor scuffles when police tried to prevent a small number of protesters from storming the office.

A group of five pro-democracy legislators plan to resign en masse from the city's legislature, following the release of a political reform blueprint for elections in 2012, which democracy advocates say does not go far enough.

The subsequent city-wide by-elections in Hong Kong's five major districts will trigger what the liberals say amounts to a symbolic referendum on full democracy.

Beijing has already promised to allow a full-scale election in Hong Kong in 2017 for the city's leader. But recent signs, including comments by pro-Beijing figures, have suggested Beijing may only allow a power-preserving version of democracy with rules stacked against opposition candidates.

Hong Kong's mini-constitution guarantees full democracy as an "ultimate aim" but the city's seven million people now have no direct say in their leader.

Beijing remains wary of upsurges of public discontent in Hong Kong, with Chinese premier Wen Jiabao recently warning Hong Kong's bowtie-wearing leader Donald Tsang to be wary of "deep-rooted conflicts". In 2003, half a million protesters spilled onto the streets in anger at the administration of Hong Kong's then-leader Tung Chee-hwa, who resigned soon afterwards.
http://www.theepochtimes.com/n2/content/view/27313/