2014年10月23日 没有评论

1.先按照官网的提示创建应用

2.安装pyhton2.7  https://www.python.org/download/ ,django github地址: https://github.com/django/django.git 3.django 启动 (参考之前的文章)

3.Python 安装

Python:

地址:http://www.python.org/getit/

版本:python-2.7.3.amd64.msi

 

setuptools:http://pypi.python.org/pypi/setuptools#windows

ez_setup.py文件:http://peak.telecommunity.com/dist/ez_setup.py

Share
分类: 杂九杂十 标签:

双城记

2014年9月30日 没有评论
那是最美好的时代,那是最糟糕的时代;
那是智慧的年头,那是愚昧的年头;
那是信仰的时期,那是怀疑的时期;
那是光明的季节,那是黑暗的季节;
那是希望的春天,那是失望的冬天;
我们全都在直奔天堂,我们全都在直奔相反的方向--简而言之,那时跟现在非常相象,
某些最喧嚣的权威坚持要用形容词的最高级来形容它。
说它好,是最高级的;说它不好,也是最高级的。
Share
分类: 杂九杂十 标签:

Django框架(二): 视图和URL配置

2012年12月23日 没有评论

接下来像其他的入门教程一样,从hello world开始Django的应用。再实现一个页面输出中,页面的内容是靠view function(视图函数)来生成。URL定义在URLconf中。

创建视图

在mysite目录下创建一个view.py文件用来包含程序的视图。

以hello word视图为例:

 

from django.http import HttpResponse
def hello(request):
       return HttpResponse("Hello world")

 

创建URLconf

from django.conf.urls.defaults import *
from mysite.views import hello
urlpatterns = patterns('',
('^hello/$', hello),
)

访问 http://127.0.0.1:8000/hello/
就可以看到输出的内容。

在这部分,主要的知识点是正则表达式:

符号                匹配
. (dot)            任意单一字符
\d                 任意一位数字
[A‐Z]              A 到 Z中任意一个字符(大写)
[a‐z]              a 到 z中任意一个字符(小写)
[A‐Za‐z]           a 到 z中任意一个字符(不区分大小写)
+                  匹配一个或更多 (例如, \d+ 匹配一个或 多个数字字符)
[^/]+              一个或多个不为‘/’的字符
?                  零个或一个之前的表达式(例如:\d? 匹配零个或一个数字)
*                  匹配0个或更多 (例如, \d* 匹配0个 或更多数字字符)
{1,3}              介于一个和三个(包含)之前的表达式(例如,\d{1,3}匹配一个或两个或三个数字)

正如我们看到的第一个视图例子里面看到的,一个视图功能必须返回一个HttpResponse.一旦做完,Django将完成剩余的转换Python的对象到一个合适的带有HTTP头和boday的web Response(例如,网页内容)
请求的过程总结如下:
1.进来的请求传入/hello/.
2.Django通过在ROOT_URLCONF配置来决定根URLconf.
3.Django在URconf中的所有URL模式中查找第一个匹配/hello/的条目
4.如果找到匹配,将调用相应的视图函数
5.视图函数返回一个HttpResponse
6.Django转换HttpResponse为一个合适的HTTP response,为Web page显示出来

Share
分类: Python 标签:

Django框架(一): 安装部署

2012年12月20日 没有评论

Django做为python最热门的web框架,今天尝试部署。像其他的框架一样,我们也是主要学习他的命名规范和API。通过这些来完成一个网络框架的使用。使用Django的前提是对python的熟悉。

Django的安装:

我选择的从源码安装,通过github来获取源代码来进行安装。

git clone https://github.com/django/django.git

oot权限下:

python setup.py install

接下来查看Django安装是否成功哦。

import django
django.VERSION
输出:
(1, 6, 0, 'alpha', 0)

安装数据库:

Django的web应用一般都需要一种数据库来支持,再使用数据库前需要安装一个数据库驱动。
支持以下四种数据库:
PostgreSQL(http://www.postgresql.org)
SQLite3 (http://www.aqlite.org)
MySQL(http://www.mysql.com)
Oracle(http://www.oracle.com)
现在比较主流的数据库是mysql,之前接触过sqlite比较轻量级别的数据库,这里就是用mysql最为以后的开发环境。
通过源来安装Mysql,安装完成后设置root的密码。
mysqladmin -uroot -p -h localhost password passwd
如果有需要,可以修改 /etc/mysql/my.cnf 对 MySQL 进行配置。
然后启动 MySQL:
lrqrun@lrqrun:/etc/init.d/mysql start
测试是否安装成功:
lrqrun@lrqrun:~/work/Django$ mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 39
Server version: 5.5.28-0ubuntu0.12.04.3 (Ubuntu)

Copyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

Django的部署:

创建你的工作目录:
/home/username/djcode
转到你创建的目录,运行命令django-admin.py startproject mysite 这样在你的当前目录下创建一个目录  mysite
他包含以下文件:
mysite/
           manage.py
           mysite/
                      __init__.py
                      settings.py
                      urls.py
                      wsgi.py
*__init__.py : 让python把该目录当成一个开发包(即一个模块)所需的文件。这是一个空文件,一般你不需要修改它。
*manage.py :一种命令行工具,允许你以多种方式与该Django项目进行交互。键入python manage.py help,看一下它能做什么。你不需要编辑这个文件。
Usage: manage.py subcommand [options] [args]

Options:
  -v VERBOSITY, --verbosity=VERBOSITY
                        Verbosity level; 0=minimal output, 1=normal output,
                        2=verbose output, 3=very verbose output
  --settings=SETTINGS   The Python path to a settings module, e.g.
                        "myproject.settings.main". If this isn't provided, the
                        DJANGO_SETTINGS_MODULE environment variable will be
                        used.
  --pythonpath=PYTHONPATH
                        A directory to add to the Python path, e.g.
                        "/home/djangoprojects/myproject".
  --traceback           Print traceback on exception
  --version             show program's version number and exit
  -h, --help            show this help message and exit

Type 'manage.py help <subcommand>' for help on a specific subcommand.

Available subcommands:

[auth]
    changepassword
    createsuperuser

[django]
    cleanup
    compilemessages
    createcachetable
    dbshell
    diffsettings
    dumpdata
    flush
    inspectdb
    loaddata
    makemessages
    runfcgi
    shell
    sql
    sqlall
    sqlclear
    sqlcustom
    sqlflush
    sqlindexes
    sqlinitialdata
    sqlsequencereset
    startapp
    startproject
    syncdb
    test
    testserver
    validate

[sessions]
    clearsessions

[staticfiles]
    collectstatic
    findstatic
    runserver
*settings.py : 该Django项目的设置或配置。查看并理解这个文件中可用的设置类型及其默认值。
*urls.py : Django项目的URL设置。可视其为你的Django网站的目录。目前它是空的。
*wsgi: Django项目WSGI  WEB服务框架的配置文件。

运行开发服务器:

在当前目录下运行python manage.py runserver 启动服务器。
默认情况下runserver命令在8000端口启动开发服务器,且仅监听本地连接。要想更改服务器端口的话,可将端口作为命令行参数传入:
python manage.py runserver 8080
Validating models...

0 errors found
December 15, 2012 - 00:09:42
Django version 1.6, using settings 'mysite.settings'
Development server is running at http://127.0.0.1:8080/
Quit the server with CONTROL-C.

当然也可一通过指定一个IP地址来监听任意的网络接口。
python manage.py runserver 0.0.0.0:8080


Share
分类: Python 标签: ,

ClusterShell部署

2012年10月17日 没有评论

ClusterShell是一个运维工具,通过这个工具可以知道服务器集群的所有机器的即时负载情况。首先是下载ClusterShell的最新版本。

通过 apt-get install clustershell安装最新版本。

根据自己的需要修改配置文件:

#mkdir /etc/clustershell
#cp conf/* /etc/clustershell
修改配置文件:/etc/clustershell/groups
加入: swift: proxy storage_1 storage_2 storage_3 Client
现在就可以查看当前的机器的负载情况:
#clush -b -g swift "uptime"
需要用的免密码登录。
shell> cat ~/.ssh/config
Host db_1
Hostname <SERVER>
User <USER>
Port <PORT>

Host db_2
Hostname <SERVER>
User <USER>
Port <PORT>

Host db_3
Hostname <SERVER>
User <USER>
Port <PORT>
Share
分类: Linux 标签:

Linux磁盘

2012年10月12日 没有评论

首先是两个主要的配置文件  /etc/fstab   /etc/mtab .

/etc/fstab文件的作用

文件/etc/fstab存放的是系统中的文件系统信息。当正确的设置了该文件,则可以通过”mount /directoryname”命令来加载一个文件系统,每种文件系统都对应一个独立的行,每行中的字段都有空格或tab键分开。同时fsck、mount、umount的等命令都利用该程序。

 

/etc/mtab文件的作用:

记载的是现在系统已经装载的文件系统,包括操作系统建立的虚拟文件等;而/etc/fstab是系统准备装载的。直接使用mount和确定就是通过查询它而来的。

 

磁盘挂载的命令:

mount  /dev/vdb    /srv/node/vdb

umount /dev/vdb

 

磁盘的使用情况:

fdisk -l

df -lh

 

Share
分类: Linux 标签:

syslog-ng 整合swift的log文件

2012年9月24日 没有评论

1.syslog-ng 的配置

1)日志的来源:

source :(分为两类)

本地的log:

source s_src {
        system();
        internal();
};
客户机的log(本地监听的端口):
source s_net{
        tcp(ip(127.0.0.1) port(541));
}
2)过滤器:
filter f_local { facility(local0, local1, local2, local3, local4, local5,
                        local6, local7) and not filter(f_debug); };
通过设置条件过滤符合自己的log信息
3)目的log文件
本地的log文件:
destination d_swift{ file("/var/log/swift.log"); };
日志服务器:
destination d_swift{ tcp("192.168.1.10" port(514)); };
4)log路径:
log { source(s_src); filter(f_swift); destination(d_swift); };

Share
分类: Linux 标签:

Swift多节点部署

2012年9月19日 没有评论

1.安装环境

node:一台物理服务器可以运行一台或者多台服务器

proxy node:运行Proxy server,TempAuth

Storage node: 运行 Account Server,Container Server,Object Server

ring : 设置一个Swift数据到物理磁盘的图

2.部署方案:

2个物理机器(10.67.15.136 10.67.15.137)  划分4个zone   每个机器 10块 1T 的磁盘

136:proxy node +Stroage node

137:Stroage node

3.所有物理机器的基础环境

1)安装Swift 的软件基础(swift从源代码安装)

apt-get install python-software-properties

apt-get update

apt-get install python-swiftclient  openssh-server curl gcc git-core memcached python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-webob python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip

2)创建配置目录

mkdir -p /etc/swift

chown -R swift:swift /etc/swift

3)创建 /etc/swift/swift.conf加入以下内容

[swift-hash]
# random unique string that can never change (DO NOT LOSE)
swift_hash_path_suffix = `od -t x8 -N 8 -A n </dev/random`

4.Proxy node 的configure

1) 从源码安装 swift    (安装在/usr/local/openstack)
2)创建一个单独的SSL证书(可选)
cd /etc/swift
openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
3)修改memcached的配置文件,监听默认的端口。推荐配置为内部的、非公用的IP网络地址。编辑 /etc/memcached.conf
perl -pi -e "s/-l 127.0.0.1/-l 10.67.15.136/" /etc/memcached.conf
4)重新启动memcached server
service memcached restart
5)创建 /etc/swift/proxy-server.conf
[DEFAULT]
cert_file = /etc/swift/cert.crt
key_file = /etc/swift/cert.key
bind_port = 8080
workers = 8
user = swift

[pipeline:main]
pipeline = healthcheck cache tempauth proxy-server

[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true

[filter:tempauth]
use = egg:swift#tempauth
user_system_root = testpass .admin https://10.67.15.136:8080/v1/AUTH_system

[filter:healthcheck]
use = egg:swift#healthcheck

[filter:cache]
use = egg:swift#memcache
memcache_servers = $10.67.15.136:11211
6)创建 account, container 和 object ring。swift-ring-builder 命令创建相应的.builder文件会有几个参数。18代表partition的数目会到2^18幂,这个数字取决于你希望一个ring中会有多少个partition。3代表每个object的副本数。最后一个参数1,代表一个partition至少在1小时之后才能被移动。:
 
这些操作可以通过脚本实现(ring.sh):
cd /etc/swift
rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz
swift-ring-builder account.builder create 18 3 1
swift-ring-builder container.builder create 18 3 1
swift-ring-builder object.builder create 18 3 1
swift-ring-builder account.builder add z1-10.67.15.136:6002/sdb 100
swift-ring-builder account.builder add z1-10.67.15.136:6002/sdc 100
swift-ring-builder account.builder add z1-10.67.15.136:6002/sdd 100
swift-ring-builder account.builder add z1-10.67.15.136:6002/sde 100
swift-ring-builder account.builder add z1-10.67.15.136:6002/sdf 100
swift-ring-builder account.builder add z2-10.67.15.136:6002/sdg 100
swift-ring-builder account.builder add z2-10.67.15.136:6002/sdh 100
swift-ring-builder account.builder add z2-10.67.15.136:6002/sdi 100
swift-ring-builder account.builder add z2-10.67.15.136:6002/sdj 100
swift-ring-builder account.builder add z2-10.67.15.136:6002/sdk 100
 
swift-ring-builder container.builder add z1-10.67.15.136:6001/sdb 100
swift-ring-builder container.builder add z1-10.67.15.136:6001/sdc 100
swift-ring-builder container.builder add z1-10.67.15.136:6001/sdd 100
swift-ring-builder container.builder add z1-10.67.15.136:6001/sde 100
swift-ring-builder container.builder add z1-10.67.15.136:6001/sdf 100
swift-ring-builder container.builder add z2-10.67.15.136:6001/sdg 100
swift-ring-builder container.builder add z2-10.67.15.136:6001/sdh 100
swift-ring-builder container.builder add z2-10.67.15.136:6001/sdi 100
swift-ring-builder container.builder add z2-10.67.15.136:6001/sdj 100
swift-ring-builder container.builder add z2-10.67.15.136:6001/sdk 100

swift-ring-builder object.builder add z1-10.67.15.136:6000/sdc 100
swift-ring-builder object.builder add z1-10.67.15.136:6000/sdb 100
swift-ring-builder object.builder add z1-10.67.15.136:6000/sdd 100
swift-ring-builder object.builder add z1-10.67.15.136:6000/sde 100
swift-ring-builder object.builder add z1-10.67.15.136:6000/sdf 100
swift-ring-builder object.builder add z2-10.67.15.136:6000/sdg 100
swift-ring-builder object.builder add z2-10.67.15.136:6000/sdh 100
swift-ring-builder object.builder add z2-10.67.15.136:6000/sdi 100
swift-ring-builder object.builder add z2-10.67.15.136:6000/sdj 100
swift-ring-builder object.builder add z2-10.67.15.136:6000/sdk 100
# 10.67.15.137
swift-ring-builder account.builder add z3-10.67.15.137:6002/sdb 100
swift-ring-builder container.builder add z3-10.67.15.137:6001/sdb 100
swift-ring-builder object.builder add z3-10.67.15.137:6000/sdb 100

swift-ring-builder account.builder add z3-10.67.15.137:6002/sdc 100
swift-ring-builder container.builder add z3-10.67.15.137:6001/sdc 100
swift-ring-builder object.builder add z3-10.67.15.137:6000/sdc 100

swift-ring-builder account.builder add z3-10.67.15.137:6002/sdd 100
swift-ring-builder container.builder add z3-10.67.15.137:6001/sdd 100
swift-ring-builder object.builder add z3-10.67.15.137:6000/sdd 100

swift-ring-builder account.builder add z3-10.67.15.137:6002/sde 100
swift-ring-builder container.builder add z3-10.67.15.137:6001/sde 100
swift-ring-builder object.builder add z3-10.67.15.137:6000/sde 100

swift-ring-builder account.builder add z3-10.67.15.137:6002/sdf 100
swift-ring-builder container.builder add z3-10.67.15.137:6001/sdf 100
swift-ring-builder object.builder add z3-10.67.15.137:6000/sdf 100

swift-ring-builder account.builder add z4-10.67.15.137:6002/sdg 100
swift-ring-builder container.builder add z4-10.67.15.137:6001/sdg 100
swift-ring-builder object.builder add z4-10.67.15.137:6000/sdg 100

swift-ring-builder account.builder add z4-10.67.15.137:6002/sdh 100
swift-ring-builder container.builder add z4-10.67.15.137:6001/sdh 100
swift-ring-builder object.builder add z4-10.67.15.137:6000/sdh 100

swift-ring-builder account.builder add z4-10.67.15.137:6002/sdi 100
swift-ring-builder container.builder add z4-10.67.15.137:6001/sdi 100
swift-ring-builder object.builder add z4-10.67.15.137:6000/sdi 100

swift-ring-builder account.builder add z4-10.67.15.137:6002/sdj 100
swift-ring-builder container.builder add z4-10.67.15.137:6001/sdj 100
swift-ring-builder object.builder add z4-10.67.15.137:6000/sdj 100

swift-ring-builder account.builder add z4-10.67.15.137:6002/sdk 100
swift-ring-builder container.builder add z4-10.67.15.137:6001/sdk 100
swift-ring-builder object.builder add z4-10.67.15.137:6000/sdk 100
 scp -P 2222 /etc/swift/swift.cof root@10.67.15.137:/etc/swift/
 
 scp -p 2222 /etc/swift/account.ring.gz root@10.67.15.137:/etc/swift/
 scp -P 2222 /etc/swift/container.ring.gz root@10.67.15.137:/etc/swift/
 scp -P 2222 /etc/swift/object.ring.gz root@10.67.15.137:/etc/swift/
 
 swift-ring-builder account.builder rebalance
 swift-ring-builder container.builder rebalance
 swift-ring-builder object.builder rebalance

 chown -R swift:swift /etc/swift
7)启动proxy services
swift-init proxy start

5.Storage nodes 配置 configure
1)对 Storage node 的每一个存储设备, 安装 XFS 逻辑卷 
cfdisk#创建一个新的分区
reboot
mkfs.xfs -i size=1024 /dev/sda6
echo "/dev/sda6 /srv/node/sda6 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
mkdir -p /srv/node/sda6
mount /srv/node/sda6
chown -R swift:swift /srv/node
2)创建/etc/rsyncd.conf
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 10.67.15.136
[account]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/account.lock
[container]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/container.lock
[object]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock

3)修改/etc/default/rsync 中的 RSYNC_ENABLE属性
perl -pi -e 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/' /etc/default/rsync

4). 开启 rsync 守护进程
service rsync start

5). 创建 /etc/swift/account-server.conf
[DEFAULT]
bind_ip =$STORAGE_LOCAL_NET_IP
workers = 2
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:swift#account
[account-replicator]
[account-auditor]
[account-reaper]

6). 创建 /etc/swift/container-server.conf
[DEFAULT]
bind_ip =$STORAGE_LOCAL_NET_IP
workers = 2
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:swift#container
[container-replicator]
[container-updater]
[container-auditor]

7). 创建 /etc/swift/object-server.conf
[DEFAULT]
bind_ip = $STORAGE_LOCAL_NET_IP
workers = 2
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:swift#object
[object-replicator]
[object-updater]
[object-auditor]
8). 开启 storage 服务
swift-init all start

6.创建swift管理员帐户并测试
  1. Get an X-Storage-Url and X-Auth-Token:
    curl -k -v -H 'X-Storage-User: system:root' -H 'X-Storage-Pass: testpass' https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0
  2. Check that you can HEAD the account:
    curl -k -v -H 'X-Auth-Token: <token-from-x-auth-token-above>' <url-from-x-storage-url-above>
  3. Check that swift works (at this point, expect zero containers, zero objects, and zero bytes):
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass stat
  4. Use swift to upload a few files named ‘bigfile[1-2].tgz’ to a container named ‘myfiles’:
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass upload myfiles bigfile1.tgz
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass upload myfiles bigfile2.tgz
  5. Use swift to download all files from the ‘myfiles’ container:
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass download myfiles
  6. Use swift to save a backup of your builder files to a container named ‘builders’. Very important not to lose your builders!:
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass upload builders /etc/swift/*.builder
  7. Use swift to list your containers:
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass list
  8. Use swift to list the contents of your ‘builders’ container:
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass list builders
  9. Use swift to download all files from the ‘builders’ container:
    swift -A https://$PROXY_LOCAL_NET_IP:8080/auth/v1.0 -U system:root -K testpass download builders
Share
分类: Swift 标签: ,

Linux命令

2012年9月19日 没有评论

1.tailf

2.byobu

3.Ctrl + r  搜索命令

4. | grep

Share
分类: Linux 标签:

Swiftly使用

2012年9月17日 没有评论

Swiftly 是一个备用的命令行工具,提供一个不需要proxy直接访问Swift的能力。

Swiftly有两种使用方法,一种是客户端类,一个是命令行工具。

1)客户端类:

Example as a standard end user:

from swiftly.client import Client
client = Client('http://127.0.0.1:8080/auth/v1.0',
                'test:tester', 'testing')
print client.head_account()

Example as a administrator direct user:

from swiftly.client import Client
client = Client(swift_proxy=True,
                swift_proxy_storage_path='/v1/AUTH_test')
print client.head_account()

2)命令行:

swiftly -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing head  container
Share
分类: Swift 标签: ,

Swift延时删除

2012年9月14日 没有评论

1.创建配置文件 守护进程访问

/etc/swift/object-expirer.conf

[DEFAULT]
# swift_dir = /etc/swift
# user = swift
# You can specify default log routing here if you want:
# log_name = swift
# log_facility = LOG_LOCAL0
# log_level = INFO

[object-expirer]
interval = 300

[pipeline:main]
pipeline = catch_errors cache proxy-server

[app:proxy-server]
use = egg:swift#proxy
# See proxy-server.conf-sample for options

[filter:cache]
use = egg:swift#memcache
# See proxy-server.conf-sample for options

[filter:catch_errors]
use = egg:swift#catch_errors
# See proxy-server.conf-sample for options

2. 两种方式:X-Delete-At和X-Delete-After
curl -i -XPUT -H "X-Auth-Token: <token>" \ -H " X-Delete-At:  1339429105" http://<storage_url>/container
curl -i -XPUT -H "X-Auth-Token: <token>" \ -H " X-Delete-Afeter:  120" http://<storage_url>/container

 

Share
分类: Swift 标签: ,

Swift对象版本

2012年9月14日 没有评论

1.对象版本

.修改 container server config 加上 allow_versions = True

.curl的简单例子

1.创建一个包含X-Versions-Locationcontainer或者在已经存在的container加上它。

curl -i -XPUT -H “X-Auth-Token: <token>” -H “X-Versions-Location: versions” http://<storage_url>/container

     curl -i -XPUT -H "X-Auth-Token: <token>" http://<storage_url>/versions

2.创建一个object(最初的版本)

curl -i -XPUT –data-binary 1 -H “X-Auth-Token: <token>” http://<storage_url>/container/myobject

3.再创建一个新版本的object

curl -i -XPUT –data-binary 2 -H “X-Auth-Token: <token>” \

    http://<storage_url>/container/myobject

4.查看老版本的object

    curl -i -H "X-Auth-Token: <token>" http://<storage_url>/versions?prefix=008myobject/

5.删除当前版本

curl -i -XDELETE -H “X-Auth-Token: <token>” \

    http://rage_url>/container/myobject

.大数据的版本删除

# First, upload the segments

curl -X PUT -H ‘X-Auth-Token: <token>’ \ http://<storage_url>/container/myobject/1 –data-binary ’1′

curl -X PUT -H ‘X-Auth-Token: <token>’ \ http://<storage_url>/container/myobject/2 –data-binary ’2′

curl -X PUT -H ‘X-Auth-Token: <token>’ \ http://<storage_url>/container/myobject/3 –data-binary ’3′

# Next, create the manifest file
curl -X PUT -H ‘X-Auth-Token: <token>’ \ -H ‘X-Object-Manifest: container/myobject/’ \ http://<storage_url>/container/myobject –data-binary ”

# And now we can download the segments as a single object

curl -H ‘X-Auth-Token: <token>’ \ http://<storage_url>/container/myobject

Share
分类: Swift 标签: ,

Swift 功能使用

2012年8月31日 1 条评论

一.Swift的上传下载功能

1.  获取X-Storage-Token和X-Storage-Url

curl  -v -H ‘X-Storage-User: test:tester’ -H ‘X-Storage-Pass: testing’ http://127.0.0.1:8080/auth/v1.0

About to connect() to 127.0.0.1 port 8080 (#0)
*   Trying 127.0.0.1... connected
> GET /auth/v1.0 HTTP/1.1
> User-Agent: curl/7.22.0 (i686-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3
> Host: 127.0.0.1:8080
> Accept: */*
> X-Storage-User: test:tester
> X-Storage-Pass: testing
>
< HTTP/1.1 200 OK
< X-Storage-Url: http://127.0.0.1:8080/v1/AUTH_test
< X-Storage-Token: AUTH_tkb4cd489333df4b80bb9677e2a03eb045
< X-Auth-Token: AUTH_tkb4cd489333df4b80bb9677e2a03eb045
< Content-Length: 0
< Date: Fri, 31 Aug 2012 06:20:42 GMT
<
* Connection #0 to host 127.0.0.1 left intact
* Closing connection #0

2.  创建container和列出容器

curl -X PUT -D – -H “X-Auth-Token: AUTH_tkb4cd489333df4b80bb9677e2a03eb045http://127.0.0.1:8080/v1/AUTH_test/cynric     #创建cyrnic容器,红色部分是上一步返回的信息

swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list       #查看test用户的container列表

3.  上传和列出文件

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload cynric/etc/swift/proxy-server.conf #将/etc/swift/proxy-server.conf文件上传到cynric容器中

swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list cynric   #列出cynric容器中的文件

4.  下载文件

swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download cynric   #将cynric容器下的所有文件下载到本地

Share
分类: Swift 标签: ,

swift笔记

2012年8月27日 没有评论

一.架构概述

1.proxy server代理服务器

代理服务器负责Swift架构的其余组件间的相互通信。

2.ring 环

环表示存储在硬盘上的实体名称和物理位置间的映射。帐号、容器、对象都有相应的环。

3.object server 对象服务器

Share
分类: Swift 标签: ,

Swift 源码结构

2012年8月24日 没有评论

1. swift-

| -swift      #源码文件

|- account                 #account组件源码

|-common                 #基础组件源码

|-container                #container 组件源码

|-__init__.py             #初始话的包

|-obj                          #obj组件源码

|-proxy                      #proxy组件源码

|-etc               配置文件

|-bin               swift命令文件

|-locale          做本地话处理的文件

|-setup.py     安装用的文件

|-test             测试用的文件

2. 服务器处理一个请求的过程

q     <—>   Server   <——->   [miv]  < —–  >  [miv]

一个请求需要经过的步骤:

pipeline = catch_errors healthcheck cache ratelimit tempauth proxy-logging proxy-server

catch_errors:捕获错误的中间件

healthcheck:检查健康状况的中间件(couchbase里面也有这个进程)

cache : 缓存的中间件

proxy-logging:打印log的中间件

 

Share
分类: Swift 标签: ,

swift安装部署(SAIO)

2012年8月24日 没有评论

安装步骤:

Installing dependencies and the core code

  1. apt-get install python-software-properties
  2. add-apt-repository ppa:swift-core/release
  3. apt-get update
  4. apt-get install curl gcc git-core memcached python-configobj python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-webob python-eventlet python-greenlet python-pastedeploy python-netifaces

Using a loopback device for storage

  1. mkdir /srv
  2. dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000
    (modify seek to make a larger or smaller partition)
  3. mkfs.xfs -i size=1024 /srv/swift-disk
  4. Edit /etc/fstab and add
    /srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=8 0 0
  5. mkdir /mnt/sdb1
  6. mount /mnt/sdb1
  7. mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
  8. groupadd swiftuseradd swiftuseradd swift -g swiftchown swift:swift /mnt/sdb1/*
  9. for x in {1..4}; do ln -s /mnt/sdb1/$x /srv/$x; done
  10. mkdir -p /etc/swift/object-server /etc/swift/container-server /etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4 /var/run/swift
  11. chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift
  12. Add to /etc/rc.local (before the exit 0):
    mkdir -p /var/cache/swift /var/cache/swift2 /var/cache/swift3 /var/cache/swift4
    chown swift:swift /var/cache/swift*
    mkdir -p /var/run/swift
    chown swift:swift /var/run/swift
    Setting up rsync
    1. Create /etc/rsyncd.conf:
      uid = swift
      gid = swift
      log file = /var/log/rsyncd.log
      pid file = /var/run/rsyncd.pid
      address = 127.0.0.1
      
      [account6012]
      max connections = 25
      path = /srv/1/node/
      read only = false
      lock file = /var/lock/account6012.lock
      
      [account6022]
      max connections = 25
      path = /srv/2/node/
      read only = false
      lock file = /var/lock/account6022.lock
      
      [account6032]
      max connections = 25
      path = /srv/3/node/
      read only = false
      lock file = /var/lock/account6032.lock
      
      [account6042]
      max connections = 25
      path = /srv/4/node/
      read only = false
      lock file = /var/lock/account6042.lock
      
      [container6011]
      max connections = 25
      path = /srv/1/node/
      read only = false
      lock file = /var/lock/container6011.lock
      
      [container6021]
      max connections = 25
      path = /srv/2/node/
      read only = false
      lock file = /var/lock/container6021.lock
      
      [container6031]
      max connections = 25
      path = /srv/3/node/
      read only = false
      lock file = /var/lock/container6031.lock
      
      [container6041]
      max connections = 25
      path = /srv/4/node/
      read only = false
      lock file = /var/lock/container6041.lock
      
      [object6010]
      max connections = 25
      path = /srv/1/node/
      read only = false
      lock file = /var/lock/object6010.lock
      
      [object6020]
      max connections = 25
      path = /srv/2/node/
      read only = false
      lock file = /var/lock/object6020.lock
      
      [object6030]
      max connections = 25
      path = /srv/3/node/
      read only = false
      lock file = /var/lock/object6030.lock
      
      [object6040]
      max connections = 25
      path = /srv/4/node/
      read only = false
      lock file = /var/lock/object6040.lock
    2. On Ubuntu, edit the following line in /etc/default/rsync:
      RSYNC_ENABLE=true
    3. On Ubuntuservice rsync restart
    4. Starting memcached                                                                                         为了确保memcached开启的是默认端口,把之前的服务关掉在重启。netstat -vnpl             #查看端口占用情况
      service  –status-all   #查看打开的所有服务

      service memcached  start

      Configuring each node

      1. Create /etc/swift/proxy-server.conf:
        [DEFAULT]
        bind_port = 8080
        user = swift
        log_facility = LOG_LOCAL1
        
        [pipeline:main]
        pipeline = healthcheck cache tempauth proxy-logging proxy-server
        
        [app:proxy-server]
        use = egg:swift#proxy
        allow_account_management = true
        account_autocreate = true
        
        [filter:tempauth]
        use = egg:swift#tempauth
        user_admin_admin = admin .admin .reseller_admin
        user_test_tester = testing .admin
        user_test2_tester2 = testing2 .admin
        user_test_tester3 = testing3
        
        [filter:healthcheck]
        use = egg:swift#healthcheck
        
        [filter:cache]
        use = egg:swift#memcache
        
        [filter:proxy-logging]
        use = egg:swift#proxy_logging
      2. Create /etc/swift/swift.conf:
        [swift-hash]
        # random unique string that can never change (DO NOT LOSE)
        swift_hash_path_suffix = changeme
      3. Create /etc/swift/account-server/1.conf:
        [DEFAULT]
        devices = /srv/1/node
        mount_check = false
        bind_port = 6012
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift
        
        [pipeline:main]
        pipeline = recon account-server
        
        [app:account-server]
        use = egg:swift#account
        
        [filter:recon]
        use = egg:swift#recon
        
        [account-replicator]
        vm_test_mode = yes
        
        [account-auditor]
        
        [account-reaper]
      4. Create /etc/swift/account-server/2.conf:
        [DEFAULT]
        devices = /srv/2/node
        mount_check = false
        bind_port = 6022
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift2
        
        [pipeline:main]
        pipeline = recon account-server
        
        [app:account-server]
        use = egg:swift#account
        
        [filter:recon]
        use = egg:swift#recon
        
        [account-replicator]
        vm_test_mode = yes
        
        [account-auditor]
        
        [account-reaper]
      5. Create /etc/swift/account-server/3.conf:
        [DEFAULT]
        devices = /srv/3/node
        mount_check = false
        bind_port = 6032
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift3
        
        [pipeline:main]
        pipeline = recon account-server
        
        [app:account-server]
        use = egg:swift#account
        
        [filter:recon]
        use = egg:swift#recon
        
        [account-replicator]
        vm_test_mode = yes
        
        [account-auditor]
        
        [account-reaper]
      6. Create /etc/swift/account-server/4.conf:
        [DEFAULT]
        devices = /srv/4/node
        mount_check = false
        bind_port = 6042
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift4
        
        [pipeline:main]
        pipeline = recon account-server
        
        [app:account-server]
        use = egg:swift#account
        
        [filter:recon]
        use = egg:swift#recon
        
        [account-replicator]
        vm_test_mode = yes
        
        [account-auditor]
        
        [account-reaper]
      7. Create /etc/swift/container-server/1.conf:
        [DEFAULT]
        devices = /srv/1/node
        mount_check = false
        bind_port = 6011
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift
        
        [pipeline:main]
        pipeline = recon container-server
        
        [app:container-server]
        use = egg:swift#container
        
        [filter:recon]
        use = egg:swift#recon
        
        [container-replicator]
        vm_test_mode = yes
        
        [container-updater]
        
        [container-auditor]
        
        [container-sync]
      8. Create /etc/swift/container-server/2.conf:
        [DEFAULT]
        devices = /srv/2/node
        mount_check = false
        bind_port = 6021
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift2
        
        [pipeline:main]
        pipeline = recon container-server
        
        [app:container-server]
        use = egg:swift#container
        
        [filter:recon]
        use = egg:swift#recon
        
        [container-replicator]
        vm_test_mode = yes
        
        [container-updater]
        
        [container-auditor]
        
        [container-sync]
      9. Create /etc/swift/container-server/3.conf:
        [DEFAULT]
        devices = /srv/3/node
        mount_check = false
        bind_port = 6031
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift3
        
        [pipeline:main]
        pipeline = recon container-server
        
        [app:container-server]
        use = egg:swift#container
        
        [filter:recon]
        use = egg:swift#recon
        
        [container-replicator]
        vm_test_mode = yes
        
        [container-updater]
        
        [container-auditor]
        
        [container-sync]
      10. Create /etc/swift/container-server/4.conf:
        [DEFAULT]
        devices = /srv/4/node
        mount_check = false
        bind_port = 6041
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift4
        
        [pipeline:main]
        pipeline = recon container-server
        
        [app:container-server]
        use = egg:swift#container
        
        [filter:recon]
        use = egg:swift#recon
        
        [container-replicator]
        vm_test_mode = yes
        
        [container-updater]
        
        [container-auditor]
        
        [container-sync]
      11. Create /etc/swift/object-server/1.conf:
        [DEFAULT]
        devices = /srv/1/node
        mount_check = false
        bind_port = 6010
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift
        
        [pipeline:main]
        pipeline = recon object-server
        
        [app:object-server]
        use = egg:swift#object
        
        [filter:recon]
        use = egg:swift#recon
        
        [object-replicator]
        vm_test_mode = yes
        
        [object-updater]
        
        [object-auditor]
      12. Create /etc/swift/object-server/2.conf:
        [DEFAULT]
        devices = /srv/2/node
        mount_check = false
        bind_port = 6020
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift2
        
        [pipeline:main]
        pipeline = recon object-server
        
        [app:object-server]
        use = egg:swift#object
        
        [filter:recon]
        use = egg:swift#recon
        
        [object-replicator]
        vm_test_mode = yes
        
        [object-updater]
        
        [object-auditor]
      13. Create /etc/swift/object-server/3.conf:
        [DEFAULT]
        devices = /srv/3/node
        mount_check = false
        bind_port = 6030
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift3
        
        [pipeline:main]
        pipeline = recon object-server
        
        [app:object-server]
        use = egg:swift#object
        
        [filter:recon]
        use = egg:swift#recon
        
        [object-replicator]
        vm_test_mode = yes
        
        [object-updater]
        
        [object-auditor]
      14. Create /etc/swift/object-server/4.conf:
        [DEFAULT]
        devices = /srv/4/node
        mount_check = false
        bind_port = 6040
        user = swift
        log_facility = LOG_LOCAL0
        recon_cache_path = /var/cache/swift4
        
        [pipeline:main]
        pipeline = recon object-server
        
        [app:object-server]
        use = egg:swift#object
        
        [filter:recon]
        use = egg:swift#recon
        
        [object-replicator]
        vm_test_mode = yes
        
        [object-updater]
        
        [object-auditor]

      Setting up scripts for running Swift

      1.Create ~/bin/resetswift.
      #!/bin/bash
      
      swift-init all stop
      find /var/log/swift -type f -exec rm -f {} \;
      sudo umount /mnt/sdb1
      sudo mkfs.xfs -f -i size=1024 /dev/sdb1
      sudo mount /mnt/sdb1
      sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
      sudo chown <your-user-name>:<your-group-name> /mnt/sdb1/*
      mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3 /srv/4/node/sdb4
      sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log /var/log/syslog
      find /var/cache/swift* -type f -name *.recon -exec -rm -f {} \;
      sudo service rsyslog restart
      sudo service memcached restart

      2.Create ~/bin/remakerings:

      #!/bin/bash
      
      cd /etc/swift
      
      rm -f *.builder *.ring.gz backups/*.builder backups/*.ring.gz
      
      swift-ring-builder object.builder create 18 3 1
      swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1
      swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1
      swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1
      swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1
      swift-ring-builder object.builder rebalance
      swift-ring-builder container.builder create 18 3 1
      swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1
      swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1
      swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1
      swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1
      swift-ring-builder container.builder rebalance
      swift-ring-builder account.builder create 18 3 1
      swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1
      swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1
      swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1
      swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1
      swift-ring-builder account.builder rebalance

      3.Create ~/bin/startmain:

      #!/bin/bash
      
      swift-init main start

      4.Create ~/bin/startrest:

      #!/bin/bash
      
      swift-init rest start

      5.chmod +x ~/bin/*

      6.remakerings

      Getting the code and setting up test environment

      apt-get install syslog-ng
      mkdir /usr/local/sws/
      cd /usr/local/sws/
      git clone git://github.com/openstack/swift.git
      git clone git://github.com/openstack/python-swiftclient.git
      cd swift
      python setup.py develop
       cd ..
      cd python-swiftclient
      python setup.py develop
      swift-init all start
Share
分类: Swift 标签: , ,

Python单元测试

2012年8月22日 没有评论

简单实例:

import unittest            # 导入单元测试模块

from demo import Demo

class DemoTests(unittest.TestCase):  # 由TestCase派生类

def setUp(self): # 开始单元测试准备工作,初始化环境

pass

def tearDown(self): # 完成单元测试收尾,清除环境

pass

def testAdd(self): # 完成单元测试收尾,清除环境

pass

def suite():

suite = unittest.TestSuite()

suite.addTest(Demo(‘testAdd’))

return suite

if __name__ == “__main__”:

unittest.TextTestRunner().run(suite())

参考:

http://docs.python.org/library/unittest.html

http://pyunit.sourceforge.net/pyunit_cn.html

Share
分类: Python 标签:

Python数据转换操作表

2012年8月21日 没有评论
#数据并不麻烦,它们不过就是数据。如果多了,就过滤。如果不是我们要的,就映射。聚焦在数据上,摒
弃费力的劳作。
#重点是捕捉 “是什么以及为什么”,而不是 “如何做”。重点是函数的定义而不是状态机的实现。

#List排重
i = [1, 2, 3, 4, 5, 6, 7, 1, 2, 3, 5, 6, 9, 0]
# o = [0, 1, 2, 3, 4, 5, 6, 7, 9]
o = list( set(i) )

#合并List
i = [[1], [2, 3], [4, 5, 6], [1, 2, 3]]
# o = [1, 2, 3, 4, 5, 6, 1, 2, 3]
o = sum( i, [] )

#序号化序列
i = ['a', 'b', 'c']
# o = [(0, 'a'), (1, 'b'), (2, 'c')]
o = list( enumerate(i) )

#反向化序列j
i = ['a', 'b', 'c']
# o = ['c', 'b', 'a']
o = list( reversed(i) )

#把所有非List元素转为List
i = [[1], [2, 3], 4]
# o = [[1], [2, 3], [4]]
o = [ x if type(x)==type([]) else [x,] for x in i ]

#不唯一的元素
i = [1, 2, 3, 4, 5, 6, 7, 8, 1, 3, 5, 7, 8]
# o = [1, 3, 5, 7, 8, 1, 3, 5, 7, 8]
o = [ x for x in i if i.count(x)!= 1 ]

#找出List中所有长度为3的单词的所在位置
i = ['How', 'are', 'you', '?', 'Fine', '.', 'Thank', 'you', '.']
# o = [0, 1, 2, 7]
o = [ idx for idx, x in enumerate(i) if len(x) == 3 ]

#切分List
i = [1, 'hello', 5, 6, 'world', 7]
# o = [[1, 5, 6, 7], ['hello', 'world']]
o = [ [ y for y in i if type(y) == x ] for x in set([type(x) for x in i])]

#反向索引
i = [1, 1, 4, 5, 9, 6, 4]
# o = [(1, [0, 1]), (4, [2, 6]), (5, [3]), (6, [5]), (9, [4])]
o = [ ( x, [ idx for idx, y in enumerate(i) if x == y ] ) for x in set(i) ]

#取区间
i = [2, 3, 5, 7, 11, 13, 17, 23]
# o = [(2, 3), (3, 5), (5, 7), (7, 11), (11, 13), (13, 17), (17, 23)]
o = zip( i[:-1], i[1:] )

#取区间2
i = [2, 3, 5, 7, 11, 13, 17, 23]
# o = [(2, 3), (5, 7), (11, 13), (17, 23)]
o = zip( i[::2], i[1::2] )

#取区间3
i = [2, 3, 5, 7, 11, 13, 17, 23]
# o = [(2, 3, 5), (5, 7, 11), (11, 13, 17)]
o = zip( i[::2], i[1::2], i[2::2] )

#求置换矩阵
i = [['a', 'b', 'c'], ['d', 'e', 'f'], ['g', 'h', 'i']]
# o = [['a', 'd', 'g'], ['b', 'e', 'h'], ['c', 'f', 'i']]
o = zip(*i)

#矩阵乘法
i = [[1, 2, 3], [4, 5, 6]]
# o = [(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)
]
o = [ ( a, b ) for a in i[0] for b in i[1] ]

#矩阵乘法2
i = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
# o = [[1, 4, 7], [1, 4, 8], [1, 4, 9],
#[1, 5, 7], [1, 5, 8], [1, 5, 9],
#[1, 6, 7], [1, 6, 8], [1, 6, 9],
#[2, 4, 7], [2, 4, 8], [2, 4, 9],
#...
#[3, 6, 7], [3, 6, 8], [3, 6, 9]]
o = reduce( lambda x, y : [ a+[b,] for a in x for b in y ], i, [[]] )

#分段落1
i = '''\
this a phase
this a phase
this
a
phase
this a phase
'''.splitlines()
#o=['this a phase', 'this a phase', 'this\n a\n phase', 'this a phase']
o=[ idx for idx, l in enumerate(i) if not l.startswith(' ') ]
o=zip( o, o[1:]+[None,] )
o=[ '\n'.join(i[s:e]) for s, e in o ]

#分段落2
i = '''\
[sectionA]
itemA
itemB
[sectionB]
itemC
itemD
[sectionC]
[sectionD]
itemE
'''.splitlines()
# o = {'sectionA': ['itemA', 'itemB'],
#      'sectionB': ['itemC', 'itemD'],
#      'sectionC': [],
#      'sectionD': ['itemE']}
o = [ idx for idx, l in enumerate(i) if l.startswith('[') and l.endswith(']')
]
o = zip( o, o[1:]+[None,] )
o = dict([ ( i[s][1:-1], [ l for l in i[s+1:e] if l != '' ] ) for s, e in o ]
)

#生成替换Dict
i = (('a', 'b', 'c'), ('e', 'f', 'g'), ('h'))
# o = {'a': 'a', 'b': 'a', 'c': 'a', 'e': 'e', 'f': 'e', 'g': 'e', 'h': 'h'}
o = dict( sum( [ zip( x, [x[0],]*len(x) ) for x in i ], [] ) )

#合并Dict
i = [{'a': 1, 'b': 2}, {'a':
# o = {'a': [1, 3], 'b': [2,
o = set(sum([ x.keys() for x
o = dict([ (key,[ x[key] for
3, 'b': 4, 'c': 6}, {'c': 7, 'd': 8}]
4], 'c': [6, 7], 'd': [8]}
in i ],[]))
x in i if key in x ]) for key in o ])

#以'idx'为关键字合并Dict ('切分List'与'合并Dict'结合)
i = [{'a': 1, 'b': 2, 'idx': 'hello'},
{'a': 3, 'b': 4, 'c': 6, 'idx': 'hello'},
{'c': 7, 'd': 8, 'idx': 'world'}]
# o = {'hello': {'a': [1, 3], 'b': [2, 4], 'c': [6]},
#'world': {'c': [7], 'd': [8]}}
o = set([ d['idx'] for d in i ])
ods = [ [ d for d in i if d['idx'] == k ] for k in o ]
oks = [ set(sum([ d.keys() for d in od ],[])) for od in ods ]
ods = [ dict([ ( k, [ d[k] for d in od if k in d ] ) for k in ok if k!='idx']
)
           for od, ok in zip(ods, oks) ]
o = dict( zip( o, ods ) )
Share
分类: Python 标签:

无觅相关文章插件,快速提升流量